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ABSTRACT 

In 2001 Congress appropriated funds for a trial district- 
level assessment, and the National Assessment Governing Board passed a 
resolution approving the selection of five large urban school districts for 
participation in the Trial Urban District Assessment, a special project 
within the National Assessment of Educational 'Progress (NAEP) . This report 
presents, for the first time, district-level results for NAEP reading 
assessments in these school districts: Atlanta City, Georgia; Chicago, 
Illinois; Houston Independent School District, Texas; Los Angeles Unified 
School District, California; and New York City Public Schools, New York. 
Results are also included for the District of Columbia. The reading 
assessments were conducted at grades 4 and 8. Reported results are based on 
representative samples of students for the nation and the participating 
districts, although New York City data are not reported for grade 8 because 
eighth-grade participation did not meet the statistical participation 
criteria. Average scale scores for fourth graders ranged from 191 in the 
District of Columbia and Los Angeles to 206 in Houston and New York City, and 
the average score for public school students in the entire country was higher 
than the average score in each of the districts. The average score for 
central city public schools was higher than the average in each of these 
urban districts. Average scale scores for eighth graders ranged from 236 in 
Atlanta to 249 in Chicago, and again national averages were higher. In terms 
of reading achievement levels, students in the study districts generally 
performed worse than those in public schools nationwide and in central city 
public schools. The report also provides information about subgroups of 
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students identified by various background and contextual characteristics, 
including race/ethnicity, gender, and elegibility for free or reduced-price 
school lunch. Three appendices contain information about study procedures and 
analytic techniques. (SLD) 
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what is The Nation's Report Card? 

THE NATION’S REPORT CARD, the National Assessment of Educational Progress (NAEP), is a nationally 
representative and continuing assessment of what America's students know and can do in various subject areas. 
.Since 1.969, assessments have been conducted periodically in reading, malliematics, science, writing, history, 
geography, and other fields. By making objective information on student performance available to policymakers at 
the national, state, and local levels, NAEP is an integral part of our nation s evaluation of the condition and progress 
of education. Only iriformalion related to academic achievement is collected imder this program, NAEP guarantees 
the privacy of individual students and their families. 

NAEP is a congressional ly mandated project of the National Center for Education Statistics within the Instituie 
of Education Sciences of the U.S. DeparUnenl of Education, The Commissioner of Education Slalislics is 
responsible, by law, for cariying out the NAEIP project through competitive awards to qualified organizations. 

In 1988, Congress established the National A.ssessment Governing Board (NAGB) to oversee and set policy for 
NAEP. The Board is responsible for: selecting llie subject areas to be assessed; setting appropriate student 
achievement leveEs; developing assessment objectives and te.st specifications; developing a process for the review of 
llie assessment; designing the assessment methodology; developing guidelines for reporting and cli.sseminating 
NAEP results; developing standards and procedures for interstate, regional, and national comparisons; determining 
the appropriateness of all assessment items and ensuring the assessment items are free from bias and are secular, 
neutral, and non-ideoiogical; taking actions to improve the form, content, use, and reporting of results of the National 
Assessment; and planning and executing ihe initial public release of National Assessment of Educational Progress 
reports. 
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b . s 

■MB xecutive Summary 

The National Assessment of Educational Progress (NAEP) 
is the nation’s ongoing representative sample survey of 
student achievement in core subject areas. NAEP, known as 
the Nation’s Report Card, is authorized by Congress and 
adrninistered bv the National Center for Education Statistics 

j 

(NCES) of tlie Institute of Education Sciences in the U.S. 
Department of Education. NAEP regularly reports to the 
public on the educational progress of students in grades 
4, 8, and 12. 

In 2002, NAEP assessed the reading and writing 
performance of the nation’s fourth-, eighth-, and twelfth- 
grade students. NAEP also conducted assessments of 
fourth- and eighth-graders’ reading and writing in most of 
the states. 

In 2001, after discussion among NCES, the National 
Assessment Governing Board (NAGB), and the leadership 
of tlie Council of the Great City Schools, Congress 
appropriated funds for a trial district-level assessment and 
NAGB passed a resolution approving the selection of five 
large urban districts for participation in the Trial Urban 
District Assessment, a special project within NAEP. Thus, 
this report presents, for the first time, district-level results of 
Ni\EP reading assessments in five urban public-school 
districts: Atlanta City, City of Chicago, Houston ISD, Los 
Angeles Unified, and New York City Public Schools. 
Throughout this report, the districts are referred to simply as 
Atlanta, Chicago, Houston, Los Angeles, and New York City. 
The five districts participated voluntarily in the NAEP 2002 
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Trial Urban District Assessment in reading 
at grades 4 and 8. Results are also included 
in this report for the District of Columbia, 
which, in 2002 and past ye«irs, has been 
included in the main assessments with states 
and other jurisdictions. Data for public 
schools across the nation and for central city 
public schools are provided for comparison 
purposes.' The public schools also included 
charter schools, which in some cases were 
not managed by the urban school district. 

NAEP does not provide scores for 
individual students or schools. It reports 
results tor groups of students (e.g., fourth- 
graders). For each group in each table in the 
report, assessment results are described in 
one ot two \v^ys: die group’s average 
reading score on a scale from 0 to 500 or 
the percentage of students m the group 
who reached each of three achievement 
levels: Basic, Proficient, •md yidvanced. The 
Pwficient le vel for each grade is defined by 
the Governing Board as representing '^solid 
academic performance,” which demon- 
strates ""competency over challenging 
subject matter” for the grade assessed. 

Basic indicates p'.u‘tial mastery of skills 
tJiat are fundamental for proficient work. 
yidvanced denotes superior performance. 

The achievement levels ;ire performance 
standards adopted by NAGB as part of its 
statutory responsibilities. The achievement 
levels are a collective judgment of what 
students should know and be able to do for 
each grade tested. As provided by law; 
NCES, upon review of a congress ion ally 
mandated evaluation of Nh\EP, determined 
that the achievement levels are to be used 
on a trial basis and should be interpreted 



with caution. However, both NCES and 
NAGB believe that the perform'^uice stan- 
dards are useful for understanding trends in 
student achievement. They have been widely 
used by national and state officials and 
others as a common yardstick of academic 
performance. 

The results are based on representative 
samples of students for the nation ;and for 
participating districts. In order to obtain 
reliable and representative data, a large 
proportion of the selected schools and 
students must participate. All six districts 
met the NCES statistical participation 
criteria for N.:\EP samples at grade 4, but 
New York City data will not be reported for 
grade 8 because eighth-grade participation 
did not meet the criteria. 

Some students are identified by the 
school districts as students with disabilities 
and/ or limited English proficient students. 
Some of these students are excluded from 
di e as s e ss m en t, an d o the rs .ire te s te d wi th 
accommodations related to their status. 
Because the percentages of students identi- 
fied, excluded, and assessed with accommo- 
dations vary across the urban districts, that 
variability should be taken into consider- 
ation in interpreting the results and making 
comparisons (see appendix A, table A.l). 

For example, in the case of fourth -grade 
students, die percentages of students 
identified as having disabilities or limited 
English proficiency ranged from 8 to 51 
percent, the percentages of fourth-grade 
students excluded for these reasons ranged 
from 2 to 17 percent, and the percentages 
assessed with accommodations ranged from 
1 to 8 percent. At the eighth grade, the 



^ “(Neutral city* iiichules a natioually representative sample of public schools located in central cities witliiu 
metropolitan statistical areas as defined by tlie federal Office of Management and Budget. “Central City* is 
nor s\^ion\mious with “inner city** 
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percentages of students identified with 
disabilities or limited English proficiency 
ranged from 6 to 35 percent, the percentage 
of eighth-grade students excluded for these 
reasons ranged from 2 to 7 percent, and the 
percentages assessed with accommodations 
ranged from 0 to 8 percent. 

Throughout this report, differences 
between scores and between percentages are 
discussed in the text only when they are 
significant from a statistical perspective. All 
differences reported are significant at the 
0.05 level with appropriate adjustments for 
multiple comparisons. 

Overall Reading Results for 
the Urban Districts 

Average Scores 

Results for Grade 4 

■ The average scale scores for fourth- 
graders ranged from 191 in the District 
of Columbia and Los ^\ngeles to 206 in 
Houston and New York City. 

■ The average score for public-school 
students in the nation as a whole was 
higher than the average score in each of 
the urban districts, and the national 
average score in central city public 
schools was higher than the average score 
in each of the urban districts except 
Houston and New York City. 

• The average scale scores in Houston 
and New Y"ork City were higher than 
those of the other urban districts and 
were not found to differ significantly 
from each other. 

Results for Grade 8 

Results for New York City schools at grade 
8 ;u*e not reported because they did not 
meet participation rates. 



• The average scale scores for eighth- 
graders ranged from 236 in Atlimta to 
249 in Chicago. 

■ The national average scores for public 
school students and for students in the 
central city public schools were higher 
than the average score in any of the 
urban districts. 

■ The average scale scores in Chicago and 
Houston were higher than those of the 
other urban districts, but were not found 
to differ significantly trom each other. 

Reading Achievement Levels 

Results for Grade 4 

■ The percentage of fourth-graders 
performing at or above P?vjide/it rTinged 
from 10 percent in the District of Co- 
lumbia to 19 percent in New York City. 
The percentage of students performing 
at or above Proficient \\\ public schools in 
the nation was 30 percent, and for stu- 
dents in central city public schools it was 
21 percent. 

■ Any apparent differences between the 
percentages of students performing at or 
above Projicient in the urban districts were 
not found to be statistically significant. 

® The percentage of students performing 
at or above Basic ningcd from 31 percent 
in the District of Columbia to 48 percent 
in Houston. In public schools across the 
nation, 62 percent of students performed 
at or above Basic. In central city schools 
in the national sample, 51 percent 
performed at or above Basic. 

® The percentages of students performing 
at or above Basicv/ere higher in Houston 
and New York City than in the other 
urban districts. 
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Results for Grade 8 

■ The percent^iges of eighth-graders 
performing at or above Prfyident: rznged 
from 8 percent in Atlanta to 17 percent in 
Houston. Thirty-one percent of students 
in public schools in the nation and 23 
percent in centrd city public schools 
performed at or above Proficient 

■ The percentages of students performing 
at or above Proficient \Vi national public 
schools and central city public schools 
were higher than tlie percentages in each 
of the urban districts. 

■ The percentages of students performing 
at or above Pasic ranged from 42 percent 
in Atlanta to 62 percent in Chicago. 
Seventy-four percent of public school 
students in the nation and 64 percent in 
central cities performed at or above Basic. 

K The percentages of students performing 
at or above Basic in Chicago ^ind Houston 
did not differ significantly from each 
other and botli were higher tlvan the 
comparable percentages in Atlanta, the 
Distnct of Columbia, ^md Los Angeles. 

Results for Student 
Subgroups 

In addition to providing average scores and 
achievement levels for the nation, for states, 
and, in this report, for urban districts, 

NAEP reports provide results for sub- 
groups of students defined by various 
background ^ind contextual chiiracteristics 
(e.g., gender, eligibility for free or reduced- 
price lunch, and level of parents’ education). 



Gender 

Results for Grade 4 

■ Both male and female fourth-grade 
students in Adanta, Chicago, the District 
of Columbia, and Los Angeles had 
average scores that were below the 
national average score for their counter- 
parts in central city public schools. 

® Female students in the urban districts 
outscored m^ile students, on average. The 
exception was Houston, where die 
apparent difference was not statistically 
significant. 

Results for Grade 8 

® The average score for male eighth- 
graders in central city schools across the 
nation was higher than die average for 
nrAe eighth-graders in each of the urban 
districts. The average score for female 
students in all urban districts except 
Chicago was below the national average 
for female students in central city public 
schools. 

■ In all urban districts, female students had 
higher average scores than male students. 

Race/Ethnicify 

VCdiereas White students constitute 60 
percent of the national public sample at 
grade 4 and 64 percent at grade 8, in the 
urban districts, \K4iite students make up a 
maximum of 15 percent of the samples at 
grade 4 (New York City) and 11 percent of 
the samples at grade 8 (Chicago). Black or 
Hispanic students or both constitute majori- 
ties in the urban districts in the trial assess- 
ment. Hispanic students made up half or 
more of the sample in Houston and lx)s 
Angeles at both grades 4 and 8. Black 
fourth- and eighth-grade students made up 
more than 80 percent of the sample in both 
Atlanta and the District of Columbia. 
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Results for Grade 4 

■ In five of the six urban districts in which 
a reliable comparison could be made, 
White fourth-graders had higher average 
scores than their Black and Hispanic 
peers. In Chicago, Hispanic students had 
higher average scores th'^in Black students. 

■ The average scores for Black students in 
Chicago, the District of Columbia, and 
Los Angeles were lower than the national 
average for Black students in central city 
public schools. 

® The average scores for Hispanic students 
in Chicago, Houston, and Ixis Angeles 
were lower thm rlie national average 
for Hispanic students in central city 
public schools. 

■ The average score for Asian/Pacific 
Islander students in New York City was 
higher than the national average for 
Asian/Pacific Islander students in central 
city public schools. 

Results for Grade 8 

■ White eighth-graders had higher average 
scores than Black eighth-graders in 
Atlanta, Houston, iind Los Aigeles. The 
apparent difference in Chicago was not 
found to be statistically significant, and 
the sample si 2 e in the District of Colum- 
bia was insufficient to permit a reliable 
comparison. 

■ The average scores for Black students in 
Atlanta, the District of Columbia, and 
Los Aigeles were lower than the national 
average for Black students in central city 
public schools. 

■ The average score for Hispanic students 
in Los Angeles was lower than the 
national average for Hispanic students in 
central city public schools. 



• The average score for White students in 
Houston was higher than the national 
average for White stxidents in central city 
public schools. 

Eligibility for 

Free/Reduced-Price Lunch 

The federal program providing free /re- 
duced-price school lunch is administered by 
the LIS. Department ot Agriculture (USDA) 
for children near or below the poverty line. 
Eligibility is determined by the USDAs 
Income Eligibility Guidelines (http:// 
www.fns.usda.gov/cnd/IEGs&NAPs/ 
IEGs.htm). At grade 4, the percentages 
of students in the urban districts eligible 
for free /reduced-price lunch ranged from 
72 percent in Houston to 88 percent in 
Chicago. By comparison, 43 percent of 
fourth -graders in public schools nationally 
were eligible. At grade 8, the percentages 
of students eligible for free/reduced-price 
lunch in four of: the urb:in districts ranged 
from 68 percent to 84 percent. By compari- 
son, 34 percent of eighth-graders in public 
schools nationally were eligible. (Informa- 
tion on the free/reduced-price lunch data 
for eighth-graders in Los Angeles is not 
reported because these data did not meet 
reporting standards.) 

Results for Grade 4 

® In each of the urban districts, fourth- 
grade students not eligible for free/ 
reduced- price lunch had higher average 
scores tJian students who were eligible. 

■ The average scores for eligible students in 
Atlanta, Chicago, the District of Colum- 
bia, -and Los Angeles were lower than the 
national average for eligible students in 
central city^ public schools. 
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■ The average scores for students in 
Atlanta, the District of Columbia, md 
Los Angeles who were not eligible for 
the program were lower than the national 
average for students in central city public 
schools who were not eligible. 

Results for Grade 8 

■ Eighth-grade students not eligible for 
free/reduced-price lunch had higher 
average scores than eligible students 
in each of the urban districts except 
Chicago, where the apparent difference 
was not statistically significant. 

■ The average scores for eligible students 
in Atlanta and tlie District of Columbia 
were lower than the national average 
for eligible students in central city 
public schools. 

■ The average scores for students in At- 
lanta, the District of Columbia, and 
Houston who were not eligible for the 
program were lower than the national 
average for students in central city public 
schools who were not eligible. 



Parents^ Highest Level 
of iducation 

Eighth -grade students who participated in 
the Trial Urban District Assessment were 
asked to indicate the highest level of educa- 
tion they thought that their parents had 
completed. Five response options were 
offered: did not finish high school, gradu- 
ated from high school, some education after 
high school, graduated from college, and 
‘T doritknow.’^ 

® In comp<u*ison with the other urban 
districts, the District of Columbia had the 
highest percentage of eighth-graders (40 
percent) who reported that at least one 
parent had gniduated from college. 

■ In each of the urban districts, the per- 
centage of students who reported that at 
least one parent had graduated from 
college was lower than that of public 
schools nationally. 

ii For students who reported that at least 
one parent graduated from college, the 
average scores for students in Atlanta, 
Chicago, tlie District of Columbia, and 
Los Angeles were lower than the national 
average for students in centnil city public 
schools. 
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Introduction 




Overview of fhe 

Trial Urban District Assessment in Reading 2002 

How are America’s schools doing? The answer is of deep 
concern to our nation. So it is very important to have 
reliable information. Witli this report, some of the largest 
urban public-school districts in the United States wiU have a 
new source of reliable data about their students with 
comparisons to public schools nationally and central city 
schools. This report presents results from the 2002 
National Assessment of Educational Progress (NAEP) 

Trial Urban District Assessment of their fourth- and 
eighth-grade students’ reading. 

Brief History of the 

National Assessment of Educational Progress 

For more tlian tliirty years NAEP has been providing 
reliable information about American students’ achievement. 
The purpose of NAEP reports is to inform educators, 
policy makers, parents, and tlie public. NAEP (also Imown 
as the Nation’s Report Card) regularly and systematically 
collects, analyzes, and reports valid and reliable information 
about what American students know and can do in a 
variety of subject areas. NAEP assesses representative 
national samples of fourtli-, eighth-, and twelfth-grade 
students, and representative samples of students in a 
number of states and other jurisdictions such as die 



^ Ccutr:d city is defined in chapter 2 and more completely in the '^Type of Location” 
section of Appendix C, Centra! city includes nationally rep resej native public schools 
located in central cities within metropolitan statistical areas as defined by the federal 
Office of Mjiuagemeut and Budget. It is not synonymous with ‘^kmer city.” 
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District of Columbia and tlie Department 
of Defense schools (domestic and over- 
seas). 

N^^EP is a continuing expression of the 
nation’s concern with students’ learning 
Congress authorized N^\EP in 1969. The 
National Center for Education Statistics 
(NCES), one of three centers within the 
U.S. Department of Education’s Institute 
of Education Sciences (lES), is responsible 
for conducting the Ni\EP. ^Vn independent 
body, the National Assessment Governing 
Board (NAGB), provides policy oversight 
for N.\EP. 

NAEP does not report results for indi- 
vidual students or schools. In fact, for its 
first two decades, N^AEP reported results 
only for the nation as a whole and for 
subgroups within the nation (e.g., for 
female students and male students). Then, 
in 1988, Congress authorized a trial of 
state-level assessment. So, in 1990, NAEP 
began collecting and providing data for 
states on mathematics performance on 
representative samples of students in 
participating states. 

The trial of state-level assessment was 
clearly successful. By 1994, NAEP was 
reporting to many states on their students’ 
performance in mathematics, reading, 
writing, and science. In 2002, N^VEP 
assessed representative samples of students 
in most of the states and also in several 
U.S. jurisdictions. What is new about this 
report is that it presents the results of 
assessments in each of six participating 
urban districts. 



The national- and state-level reports of 
the N/\EP 2002 assessment are available 
on the NAEP web site: http:// 
nces.ed.gov/ nationsreportcard/. They 
present nationwide results for grades 4, 8, 
and 12, and results for grades 4 and 8 
within the states or jurisdictions that 
participated in tlie state- level assessment. 
The national :md state reports also compare 
the results to the results of previous N^^VEP 
reading assessments. 

Background of the NAEP 
Trial Urban District Assessment 
in 2002 

There has been '^in ongoing interest in 
NAEP reports by school district '^ind even 
by school building, but NU\EP does not 
assess large enough numbers of students to 
provide reliable results at those levels. 

The District of Columbia is an exception. 
Since it is not witliin a state, Ni\EP has 
sampled and analyzed its school district as 
the equivalent of a state. 

Interest in reporting results by district 
increased with the passage of the No Child 
Left Behind Act.“ Some large urban school 
districts, such as Los Angeles and New 
York, have enough students to meet NAEP 
requirements for sample size in reporting. 

In 2001, after discussions among the 
National Center for Education Statistics, 
the National Assessment Governing Board, 
and the leadership of the Council of the 
Great City Schools, Congress appropriated 
funds for the Trial Urb^m District Assess- 
ment. NAGB passed a resolution approving 
the selection of five large urban districts 
for participation in tlie Trial Urban District 
Assessment."^ 



^ No Child Left Beliiiid .^ct of 2001, Pub. L No. 107-1 10, 115 Stat. 1425 (2002). 

^ National x'\ssessmeiit Goveniing Board. (2001, November 20). Mimites of the November 15—17, 2001 NAGB 
Meeting: Committee Reports imd Board Actions. Presented at the November 2001 NAGB meeting, Washington, DC. 
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The trial design cdls for larger-than- 
usual sample si 2 es within the districts, 
making reliable district-level data possible. 
Tlie assessment will allow these districts to 
make valid comparisons among themselves 
for the first time. Although individual 
states’ own assessments may report data by 
district md school, their results are based 
on different scores, scales, and test designs, 
so up until now, districts have not been 
able to compare themselves to districts in 
other states. 

By undertaking the Trial Urban District 
Assessment in reading and writing in 2002, 
NAEP continues a tradition of carefully 
extending its service to education, while 
preserving the rigorous sampling, scoring, 
and reporting procedures that have charac- 
terized the national and state assessments. 
The samples were large enough to provide 
data on subgroups within the districts, such 
as female stxidents or Hispanic students. 
The 2002 data can serve the districts as a 
benchmark for study of changes in the 
performance of all their students and of 
particular subgroups of students. 

N^\EP gathers contextual data about in- 
and out-of-school experiences and socio- 
economic factors from background ques- 
tionnaires given to students, teachers, :ind 
school administrators. These data for the 
reporting districts are available on the 
N^\EP web site (http://nces.ed.gov/ 
na tio n s re p o rtca rd / n ae p data /) . 



Selection of 
Trial Urban Districts 

Representatives of tlie Council of the 
Great City Schools worked with the staff 
of NAGB to identify districts for the trial 
assessment. Districts were selected tliat 
permitted testing of the feasibility of 
conducting N^\EP over a range of chvirac- 
teristics, such as district size, minority 
concentrations, federal program pvirticipa- 
tion, socioeconomic conditions, and 
percentages of students with disabilities 
and limited English proficient students. 

This report presents data for the following 
participating urban districts: Atlanta City, 
Chicago School District 299, Houston 
Independent School District, Los /Vngeles 
Unified, New York City Public Schools, 
and Washington, DC. All these p:articipat- 
ing districts are located in central cities. 

Overview of the NAEP 
2002 Reading Assessment 

What Wos Assessed? 

Each NAEP assessment has objectives 
described in a ‘Tramework” — ^a document 
that specifies the important content and 
process virevis to be measured and the types 
of questions to be included.'^ NAGB directs 
a process for specifying these frameworks. 
Teachers, curriculum specialists, subject- 
matter specialists, local school administra- 
tors, parents, -and members of the general 
public participated actively in the develop- 
ment of the reading framework. 

The NAEP 2002 Trial Urban District 
Assessment used tlie same reading frame- 
work as the national and state assessments. 
ITiis framework hvis guided the N^AEP 
reading -assessments since 1992. 



■* National Assessment Governing Board. (2002). Bjuding Framework for 2005 National Assessment of Educational 
Progress. Wasliington, DC: Aiitlior. 
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llie framework is founded on a body of 
educational research that defines reading as 
an interactive and constructive process 
involving the reader, the text, and the 
context of the reading experience. Reading 
involves the development of an under- 
standing of text, thinking about die text in 
different ways, and using a variety of text 
types for different purposes. 

Tlie framework calls for assessing 
reading in three contexts: reading tor 
literary experience, reading to gain informa- 
tion, :ind reading to perform a task. Each 
context for reading is associated with a 
range of different types of texts that are 
included in the NAEP reading assessment. 
All three contexts for reading ;ire assessed 
at grade 8, but reading to perform a task is 
not assessed at grade 4, 

As readers attempt to develop under- 
standing of text, diey focus on general 
topics or themes, interpret iind integrate 
ideas, make connections to background 
knowledge and experiences, and examine 
the content and structure of the text. The 
framework accounts for these different 
approaches to understanding text by 
specifying four ''aspects of reading” that 
represent the types of comprehension 
questions asked of students. 



The NAGB framework for die NAEP 
reading assessment lists the four aspects of 
reading as follows: 

• Forming a general understanding: To form a 
general understanding, the reader must 
consider the text as a whole and provide 
a global understanding of it. 

• Developing interpretation: lb develop an 
interpretation, the reader must extend 
initial impressions to develop a more 
complete understanding of what was 
read. 

• Making reader! text connections: To make 
reader/text connections, the reader must 
connect information in the text with 
Imowledge and experience. 

• Examining content and stmcture: Examining 
text content and structure requires 
criticiilly evaluating, comparing and 
contrasting, and understanding the effect 
of such features as irony, humor, and 
organi/:ation. 

All four aspects of reading are assessed 
at all tliree grades within each context of 
reading. Further details on the reading 
framework are available at http:// 
www.nagb.oig/ pubs/pubs.html. 
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The assessment contains reading materi- 
als that were drawn from sources com- 
monly available to students both in and out 
of the school environment. These materials 
were considered to be representative ot the 
types of reading experiences typically 
encountered by students. Each student in 
the Trial Urbiin District Assessment (as 
well as the national and state assessments) 
was asked to complete two 25-minute sets 
of questions, each consisting of a reading 
passage and comprehension questions 
associated with it. A combination of 
multiple-choice and constructed-response 
(short and extended written responses) 
questions was used to assess students’ 
understanding of the passages. The pas- 
sages and associated questions followed a 
distribution specified by the framework. 
Example NAEP reading passages and 
questions that hav'e been released to the 
public, along with student pertormance 
dam by state, are available on the NAEP 
web site (littp://nces.ed.gov/ 
nation srep o rtcard / i tmrls /) . 

Sampliiig: Who Was Assessed? 

Tlae NAEP 2002 reading assessment was 
administered to fourth-, eighth-, and 
twelfth-graders at the nabonal level and to 
fourth- and eighth-graders at the state level. 
For the NAEP state assessments, a target 
for each jurisdiction is a sample of 100 
schools and 3,000 students, except in small 
or sparsely populated jurisdictions. The 
sample of schools and students is chosen in 
a tv^^o-stage sampling process. First, the 
sample of schools is selected using prob- 
ability sampling methods. Then, within the 
participating schools, random samples of 
students are chosen. Only public schools 
are reported in state and jurisdiction 



reports. In order to obtain a representative 
sample of students for reporting national 
and state or jurisdictional results, approxi- 
mately 140,000 fourth-graders from 5,500 
schools, 115,000 eighth-graders from 4,700 
schools, and 15,000 twelfth -graders from 
700 schools were sampled and assessed. 
The public schools also included charter 
schools, which in some cases were not 
managed by the urban school district. 
Details of sampling procedures are avail- 
able in appendix C. ITie national and state 
results in 2002 draw on common samples. 
Tlie national results include the results 
from the states and trial urban districts, 
weighted appropriately to represent the 
student population. 

Svimpling for the Trial Urban District 
Assessment was modeled on die procedure 
for sampling states. The number of partici- 
pating schools ranged from 38 to 76 per 
district in the fourth grade and from 15 to 
69 per district at the eighth grade. The 
number of participating students per 
district ranged from 947 to 2,100 at the 
fourth grade and from 1,110 to 1,778 
at the eighth grade. 

The overall participation rates tor 
schools and students in the national, state, 
and trial urban district assessments must 
meet statistical guidelines established by 
NCES and NAGB in order for assessment 
results to be reported publicly. Data are not 
reported to the public for a state, jurisdic- 
tion, or urh.in district that participates but 
does not meet minimum piirticipation rate 
guidelines. For more information about 
participation guidelines, see standards for 
sample participation and reporting of 
results in appendix C. 
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As with the national svimples, the urban 
district samples were weighted to -allow for 
valid inferences about the populations of 
interest. Participation rates for the jurisdic- 
tions and urban districts were calculated 
the same way as rates were computed for 
the nation. 

,N^\EP endeavors to assess all students 
selected in the random sampling process, 
including students with disabilities (SD) 
and students who are classified by their 
schools as limited English proficient (LEP). 
The percentages of students classified as 
SD or LEP in all participating states and 
jurisdictions are available in m interactive 
database at the NAEP web site at http:/ / 
nces.ed.gov/ nationsreportcard/ naepdata/. 
Information on SD and LEP students for 
the trial urban districts appears in appendix 
A. Percentages of students identified as 
limited English proficient, particularly 
at grade 4, appe;ir much higher in some 
districts (Houston and Los i^Vngeles) than 
in the nation. 

Some students sampled for participation 
in Ni\EP can be excluded from the sample 
according to carefully defined criteria. 
Students who cannot be meaningfully 
assessed, or students for whom N^\EP 
cannot supply tlie lociilly required testing 
accommodation, can be excused from the 
assessment. Such students are assumed not 
to be part of the population groups from 
which NAEP reports assessment results. 

It is important to note that, guided by the 
student’s Individualized Education Pro- 
gram (lEP), ;is well as eligibility for '‘sec- 
tion 504” services, school personnel make 
decisions on whether students with disabili- 
ties should be included in the assessment.^ 



They also decide, based on N^AEP’s guide- 
lines, whether to include an LEP student. 
The guidelines ask them to judge the 
student’s ability to participate in the assess- 
ment in English as well as to consider the 
number of yeiirs the student has been 
receiving instruction in English. 

Percentages of students excluded from 
Ni\EP may vary considerably across 
states/districts -and, within a state or 
district, from one year to another. If there 
are great differences between groups in the 
rates of exclusion, compiirisons between 
those groups may be affected. If a higher or 
low'er percentage of sai dents in one district 
or one state or one year did not take the 
test for reasons of disability or limited 
English proficiency, or if a higher propor- 
tion required accommodations for disabili- 
ties, comparisons of saident performance 
results with similar groups (states, districts, 
or years) should be interpreted with caution. 

For example, the percentage of fourth- 
grade students identified as having disabili- 
ties and or limited English proficiency 
ranged from 19 to 30 percent in public 
schools in the nation, central cities and 
three of the six districts. Adanta w'as below 
the national average, with only 8 percent of 
its fourth-grade students identified in these 
groups; iind Houston and Los Angeles were 
above the national average, with 43 and 51 
percent, respectively, of their fourth-grade 
students identified in these groups (in both 
of these cases the majority of the identified 
students had limited English proficiency). 
Given that many students identified in 
these groups participate in the assessment, 
the percentage of fourth -grade students 
who were excluded ranged between 7 and 9 



^ Sectiou 504 of the Rehahililaliou Act of 1973 is a civil rights law desigued to prohibit discriiiiiiialion on die basis 
of disability iu programs and activities, iuchiding edncariou, tliat receive federsil financial assistance. 
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in public schools across the nation, in 
central cities, and in four of the six public 
school districts included in this report. In 
contrast to this, in Atlanta only 2 percent 
of the fourth-grade students were excluded 
from the assessment because they had 
disabilities (1 percent) or were limited 
English proficient (1 percent). At the 
same time, 17 percent of the fourth-grade 
students in Houston were excluded from 
the assessment, with 16 percent excluded 
because they had limited English 
proficiency. 

In the eighth grade, the percentage of 
students identified with disabilities and/ or 
limited English proficiency ranged from 18 
to 22 percent in public schools across the 
nation, central cities, and two of the five 
districts. Atlanta was below the national 
average, with only 6 percent of its eighth - 
grade students identified in these groups; 
and Houston and Los ^\ngeles were -above 
the national average, with 27 and 35 
percent, respectively, of their eighth-grade 
students identified in these groups (in both 
of these cases the majority of the identified 
students had limited English proficiency). 
Ihe percentages of eighth-grade students 
excluded from the assessment were more 
similar across jurisdictions. The exclusion 
rates for the nation, central city public 
schools, and four of the five districts 
ranged from 5 to 7 percent. In contrast, 
Atlanta excluded only 2 percent of its 
eighth-graders. 

In both grades the percentage of stu- 
dents assessed with accommodations 
ranged from 0 or 1 percent to 8 percent 
across jurisdictions. The variability in the 
identification, exclusion, and accommoda- 
tion rates should be taken into consider- 
ation in interpreting the results and making 
comp;arisons (see appendix A, table A.l). 
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How is Student Performance 
Reported? 

Results from the NhAEP reading assessment 
are presented in two ways; as scale scores 
on a scale of 0--500 -and -as percentages of 
students performing at different achieve- 
ment levels (Bask\ Pfvjuienr, Tind Adi meed). 
The results of student performance are 
reported for various groups of students 
(e.g., fourth-grade female students or 
students who took the assessment in 
different ye;irs). 

Throughout this report, comparisons are 
made among districts and between districts 
and public schools in the nation, as well as 
between districts and central city public 
schools in the nation. (See '^NAEP Report- 
ing Groups” in appendix C for details on 
how central city public schools were 
defined.) The significance of differences in 
performance between groups of students 
that are reported here are b-ased on smtisti- 
cal tests. The tests consider both the size 
of differences between averages or percent- 
ages and the standard error of those statis- 
tics. Every test score has a standard error — 
a range of a few points plus or minus the 
score — due to sampling error and measure- 
ment error. Statistical tests are used to 
determine whetlier the difterences between 
average scores are significant. Only statisti- 
cally significant differences are cited in this 
report. 

The reader is cautioned to rely on the 
reported differences in the text ;ind tables, 
which are statistically signific;int, rather 
than on the -app-arent magnitude of any 
difference. The standard errors are avail- 
able on the NAEP web site (http;/ / 
www.nces.ed.gov/ nationsreportcard/ 
naepdata). 
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Scale Scores 

Student performance is reported as an 
average score b 2 ised on the N^-\EP reading 
scale, which ninges from 0 to 500 and is 
linked to the conresponding scales in 1992, 
1994, and 1998. The average scale score 
reflects the ovenill reading performance of 
a particular group of students. The overall 
composite scale was developed by weight- 
ing each of the separate reading subscales 
(two for fourtli grade and three for eighth 
grade, reflecting each of the three above- 
mentioned reading contexts) based on its 
relative importance in the Ni\EP reading 
framework. Ihis composite scale is the 
metric used to present the average sc;ile 
scores and selected percentiles used in 
NAEP reports. More information on 
N^\EP scales is available in appendix C. 

Achievement Levels 

Student reading performance is also re- 
ported in terms of three achievement 
levels: Basic, Pivjide/il, and Advanced. 

Results based on achievement levels are 
expressed in terms of the percentage of 
students who attained each level. The three 
achievement levels are defined as follows: 

• Basic: Tliis level denotes partial mastery 
of prerequisite knowledge and skills diat 
are fundamental for proficient work at 
each grade. 

• Pfvjicient. This level represents solid 
academic performance for each grade 
assessed- Students reaching this level 
have demonstrated competency over 
challenging subject matter, including 
subject-matter knowledge, application of 
such knowledge to real-world situations, 
and ^malytical skills appropriate to the 
subject matter. 



• Advanceck This level signifies superior 

performance. 

The achievement levels are performance 
standards adopted by NAGB as part of its 
statutory responsibilities. The levels repre- 
sent collective judgments of what students 
should know and be able to do for each 
grade tested. They are based on recommen- 
dations of broadly representative panels of 
classroom teachers, education specralists, 
and members of the general public. As 
provided by law, the National Center for 
Education Statistics (NCES), upon review 
of congressionally mandated evaluations of 
NAEP, hiis determined that the achieve- 
ment levels are to be used on a trial basis 
until it is determined that the achievement 
levels are ^‘re:isonable, valid, and informa- 
tive to the public.''^ However, both NCES 
and NAGB believ'^e these performance 
standards are useful for understanding 
trends in student achievement. They have 
been widely used by national and state 
officials as a common yardstick for aca- 
demic performance. The reading achieve- 
ment level descriptions for grades 4 and 8 
are summarized in figures 1.1 and 1.2. The 
score ranges for the NAGB achievement 
levels on the NAEP scale are as follows: 
Grade 4, Basic, 208-237; Proficient 238-267; 
Advanced, 268 and above; Grade 8, Basic, 
243—80; Proficient, 281—322; and Advanced, 
323 and above. 



® No Child I.efr. Behind Act ot 2001, Pub. L. No. 107-110, 115 Star 1425 (2002). 
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Cautions in Interpretations 

The NAEP reading scale makes it possible 
to ex'<imine relationships between students’ 
performance and v;irious background 
factors measured by NAEP. However, a 
relationship that exists between achieve- 
ment and another variable does not reveal 
its underlying cause, which may be influ- 
enced by a number of other variables. 



Similarly, the assessments do not reflect the 
influence of unmeasured variables. The 
results are most useful when they are 
considered in combination with other 
knowledge about the student population 
and the educational system, such as trends 
in instruction, chiuiges in the school-age 
population, and societal demands and 
expectations. 



Figure 1.1 Descriptions of NAEP reading achievement levels, grade 4 



Grade 4 

Athicveinent Levels 



Bask Fourth-grade students performing at the Basic level should demonstrate on understanding of the 

(208) overall meaning of what they read. When reading text appropriate for fourth graders, they should be 

able to make relatively obvious connections between the text and their own experiences, and extend 
the ideas in the text by making simple inferences. 

For example, when reading literory text, they should be oble to tell what the story is generolly 
about— providing details to support their understanding— and be able to connect aspects of the 
stories to their own experiences. 

When reading tnformatioflal text, Mc-level fourth groders should be oble to tell whot the 
selection is generally about or identify the purpose for reading it, provide detoils to support their 
understanding, and connect ideas from the text to their background knowledge and experiences. 

Proficient Fourth-grade students performing at the /’roZ/oen/ level should be able to demonstrote on overoll 

(238) understanding of the text, providing inferential as well as literal information. When reading text 

appropriote to fourth grade, they should be able to extend the ideas in the text by moking 
inferences, drawing conclusions, and making connections to their own experiences. The connections 
between the text and what the student infers should be clear. 

For example, when reading literary text, Profkient-\en\ fourth graders should be oble to summorize 
the story, draw conclusions about the characters or plot, and recognize relationships such as cause 
and effect. 

When reading informotional text, Proficieni-kye\ students should be able to summorize the 
information and identify the author’s intent or purpose. They should be able to draw reosonoble 
conclusions from the text, recognize relationships such as cause and effect or similarities and 
differences, and identify the meaning of the selection’s key concepts. 

Advamed Fourth-grade students performing at the Advanced level should be able to generalize about topics in 

(268) the reading selection and demonstrate an awareness of how authors compose and use literary 

devices. When reading text appropriate to fourth grade, they should be able to judge texts critically 
and, in general, give thorough answers that indicate careful thought. 

For example, when reading literary text, >1(/vonce{/-level students should be able to make 
generalizations about the point of the story and extend its meaning by integrating personal 
experiences and other readings with ideas suggested by the text. They should be able to identify 
literary devices such as figurative language. 

When reading informational text, 4(/vonce(/*level fourth graders should be able to explain the 
author’s intent by using supporting material from the text. They should be able to make critical 
judgments of the form and content of the text and explain their judgments clearly. 

SOURCE: NiiliciidAuasmemGoveniingBcicrd. (2002). of EiiKitei/f^iSress.WsNngton,DC:/yW^ 
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Figure 1 .2 Descriptions of NAEP reading achievement levels, grade 8 



6rod&8 

Adrievement Levels 



Bask 

(243) 


Eighth^grode students performing at the 8o5/( level should demonstrate a literal understanding of what 
they read and be able to make some interpretations. When reading text appropriate to eighth grade, 
they should be able to identify specific aspects of the text that reflect the overall meaning, extend the 
ideas in the text by making simple inferences, recognize and relate interpretations and connectians 
among ideas in the text to personal experience, and draw conclusions based on the text. 

For example, when reading literary text, 5o5/(*level eighth graders should be able to identify themes 
and make inferences and logical predictions about aspects such as plot and characters. 

When reading Informational text, they should be able to identify the main idea and the author's 
purpose. They should make inferences and draw conclusions supported by information in the text. They 
should recognize the relationships among the facts, ideas, events, and concepts of the text (e.g., 
cause and effect, order). 

When reading practical text, they should be able to identify the main purpose and make predictions 
about the relatively obvious outcomes of procedures in the text. 


Profkient 

(281) 


Eighth-grade students performing at the Proficieniknl should be able to show an overall understand- 
ing of the text, Including Inferential as well as literal informotion. When reading text appropriate to 
eighth grade, they should be able to extend the ideas in the text by making clear inferences from it, 
by drawing conclusions, and by making connections to their own experiences— including other reading 
experiences. Proficient elqkh graders should be able to identify some of the devices authors use in 
composing text. 

For example, when reading literary text, students at the Proficieniknl should be able to give details 
and examples to support themes that they identify. They should be able to use implied as well as 
explicit information in articulating themes; to interpret the actions, behaviors, and motives of 
characters; and to Identify the use of literary devices such as personification and foreshadowing. 

When reading Informational text, they should be able to summarize the text using explicit and 
implied information and support conclusions with inferences based on the text. 

When reading practical text, Proficient-knl students should be able to describe its purpose and 
support their views with examples and details. They should be able to judge the importance of certain 
steps and procedures. 


Advanced 

(323) 


Eighth-grade students performing at the Advancedknl should be able to describe the more abstract 
themes and ideas of the overall text. When reading text appropriate to eighth grade, they should be 
able to analyze both meaning and form and support their analyses explicitly with examples from the 
text, and they should be able to extend text information by relating it to their experiences and to world 
events. At this level, student responses should be thorough, thoughtful, and extensive. 

For example, when reading literary text, Advanced-kn\ eighth graders should be able to make 
complex, abstract summaries and theme statements. They should be able to describe the interactions 
of various literary elements (i.e., setting, plot, characters, and theme) and explain how the use of 
literary devices affects both the meaning of the text and their response to the author's style. They 
should be able to critically analyze and evaluate the composition of the text. 

When reading Informational text, they should be able to analyze the author's purpose and point of 
view. They should be able to use cultural and historical background information to develop perspectives 
on the text and be able to apply text information to broad issues and world situations. 

When reading practical text, Advanced-kve\ students should be able to synthesize information that 
will guide their performance, apply text information to new situations, and critique the usefulness of the 
form and content. 
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M Average Scale Score and 
^ Achievement Level Results 

for the Trial Urban District Assessment 

This chapter presents the NAET 2002 reading results for 
the five urban districts that participated in the Trial Urban 
District Assessment and for the District of Columbia at 
grades 4 and 8. Average scores are reported on die N AEP 
reading scale, which ranges from 0 to 500, and in terms 
of percentages of students reaching the three reading 
achievement lev'^els — J^asic, Proficient, and Advanced. y\ll 
comparisons in this report have been tested for statistical 
significance. 

Urban District Scale Score and Percentile Results 

Table 2.1 shows the overall performance of fourth- and 
eighdi-grade students in the urban districts that 
participated in die 2002 reading assessment In order to 
provide a context for these data, table 2.1 also displays the 
results for students attending public schools in the nation as 
a whole, as well as for pubhc schools located in central cities 
across the nation. In table 2.1 and subsequent tables and 
figures in this report, a double asterisk marks district 
statistics (average scores or percentages) that were found to 
be significandy different from the comparable statistic in 
public schools in the nation, and a single asterisk (*) marks 
district statistics that were significandy different from those 
of public schools in central cities. Following standards 
established by the Federal Office of Management and 
Budget, the U.S. Census Bureau defines a central city as a 
city of 50,000 people or more that is the largest in its 
metropolitan area, or can otherwise be regarded as 
“central,” taking into account such characteristics as 
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commuting patterns (see ''Type of Loca- 
tion’' in appendbc C for more detailed 
definitions of geographical areas, and see 
http:/ /wvwvxensus.gov/ population/u^^v/ 
estimates/metroarea.html for more infor- 
mation on Metropolitan Statistical -:\reas). 
The term means "a city that is central,” not 
"the central part of a city” or "the inner 
city.” Central cities are defined for this 
report as including large and midsize cities. 
Note that central cities encompass wider 
areas than what is commonly referred to as 
"the inner city.” The five participating 
urban districts, Atlanta City, Chicago 
School District 299, Houston ISD, Los 
Angeles Unified, and New York City Public 
Schools, and Washington, DC vire all 
located in central cities. 

i\Jl students in these districts attended 
schools in central cities except in the 
Houston and Los Angeles districts, where 
some students included in the study at- 
tended schools located in the urban fringe. 
These included 6 percent and 19 percent of 
fourth-grade students in Houston and Los 
Angeles respectively, :md 24 percent of 
eighth-grade students in Los ^^Vngeles. 

The first column in table 2.1 presents the 
average score on the N^AEP reading scale. 
At grade 4, the average scores ranged trom 
191 for the District of Columbia and Los 
iYigeles to 206 for Houston and New York 
City. The average score for fourth-grade 
students in public schools in the nation was 
217; for students in central city public 
schools nationally the average was 208. 



At grade 8, the average scale scores ranged 
from 236 for students in Atlanta to 249 for 
students in Chicago. The average score for 
eighth-grade students in public schools in 
the nation was 263; for students in central 
city public schools, the average was 254. 
Data for New York City at grade 8 do not 
appear because the district did not meet the 
required 70 percent school piirticipation 
rate (see appendix C, Standards for Sample 
Participation and Reporting of Results). 

The remaining columns in table 2.1 show 
the scores at selected percentiles. Each 
percentile indicates the percentage of 
students in a given jurisdiction whose 
scores fell below a particular point on the 
N^AEP reading scale. For example, for 
Adanta at grade 4, the score at die 75th 
percentile was 219, indicating that 75 
percent of fourth-grade students scored at 
or below 219. Looking at different percen- 
tiles makes it possible to examine the scale 
score g'aps between higher and lower 
performing students within a district. In 
addition, comparing the scores for percen- 
tiles across districts gives a rough indica- 
tion of how their score distributions may 
differ. In subsequent yevirs, when additional 
results are available, comparing the score 
values for percentiles year to year will also 
be useful in pinpointing where changes may 
occur within a district’s score distribution. 
The corresponding standard errors for these 
percentile scores are displayed in 'appendix 
C in table C.8. 
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Table 2J Average reading scale scores and selected percentiles, grades 4 and 8 public schools: 
By urban district, 2002 









Scale score distribution 


1 




Average 
scale score 


25th percentile 


50th percentile 


75th percentile 


1 Grade 4 










Nation {Public} 


217 


194 


219 


242 


Centro! city (Public) ^ 


208** 


183** 


209** 


234 ** 


Atlanta 


195*'** 


171 *'** 


194 *'** 


219*'** 


Chicago 


193*'** 


170*'** 


194*'** 


217*'** 


District of Columbio 


191 *'** 


167 *'** 


191 *'** 


215*'** 


Houston 


206** 


183** 


206** 


229 ** 


Los Angeles 


191 


165 *'** 


190*'** 


217*'** 


New York Oty^ 


206 ** 


182** 


206** 


230** 


1 Grade 8 










Notion (Public) 


263 


242 


265 


286 


Centrol dty (Public) ^ 


254** 


232** 


256** 


278** 


Atlanto 


236*'** 


214*'** 


236*'** 


259 *'** 


Chicago 


249 *'** 


231** 


251 ’'** 


270*'** 


District of Columbio 


240*'** 


219*'** 


241 *'** 


262*'** 


Houston 


248*'** 


226** 


251 *'** 


273*'** 


los Angeles 


237*'** 


213*'** 


238 *'** 


261 *'** 



^ dsemd suffidBnt ibr reporting, the target response rote spedhed a the HAS’ guidebies was not met. 

* Significontiy different ham tentroi dty publk srhoois. 

** Signihcmitty diff^ent from notion (pa^ sdiook). 

^ For (omporison, ot fourth grade 65 percent of students in coitro! dty pubhc schools end 40 perc^t in pubhc schook notiondiy were non-White. Ako, 61 percent of students ei centred city puhk schook and 
43 percent in puhik schxk nottonoffy were eh$*ble for free/reduceditrice school lunch. 

^ For comporison, ot eighth grade 61 perext of stdents in central dty pub^c schook and 36 percent m public schook notiimaOy were non-White. Ako, 47 percent of students in centrol dty pubbe schools ond 
34 percent In public sdmok notiondly were obgiUe for kee/reducediHice school lunch. 

SOURCE: U.S. Deportment of Bfucolion, Institute of Blucation Sd^ices, Notioiol Cotter for Ethtcotix Statistics, Notiond Asse^mait of Educotioned Pre^ess (NAEP), 2002 Tiid Urbon Distrid Rxebng Assessment. 



Comparisons Among Districts 
by Average Scale Scores 

Figures 2.1 anci 2.2 display the results 
of statistical significance tests of differ- 
ences in the N.AE.P 2002 average reading 
scale scores of participating districts at 
grades 4 and 8. These figures are similar to 
mileage ch;irts on travel maps: To read 
them, find the mime of the district of 
interest (‘'target'') in the rows on the left 



side ol the figure. "ITien iollow that row 
across the columns until you reach the 
district ("comparison district") whose 
average score you wish to compare to the 
target. If the cell comp:iring the target 
district (the row variable) to the compari- 
son district is lighdy shaded (with an 
upward arrow), the average sede score of 
the target district was higher than that of 
the "comparison district" named at the top 
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of die column. ITie darkly shaded cells 
(with a downward arrow) indicate that the 
average sc:ile score of the target district 
was lower than that of the comparison 
district. Cells that are not shaded indicate 
tJiat a statistically significant difference 
between the two districts was not detected. 
For example, in figure 2.1, the third cell 
in the third row comp;ires the score at 
grade 4 in Atlanta to the average score in 
New York City. The shading in this cell 
indicates that the results of the statistical 
significance test showed the average score 
in Atlanta to be lower than that in New 
York City. 



At grade 4, the average scale score for 
students in public schools in the nation was 
higher than die average score in each urban 
district. The average score for students in 
central city public schools across the nation 
was higher than the average score in 
Adanta, Chicago, Los ^\ngeles, and the 
District of Columbia. The average scores 
for students in Houston and New York City 
were higher than those in the other districts 
and were not found to differ significantly 
from each other. Finally, the average score 
in Adanta was higher dian the average 
score in the District of Columbia. Other 
apparent differences were not statistically 
significant. 



Figure 2 A Cross-distrkf comparisons of average reading scale scores, grade 4 public schools: By urban district, 2002 



Grade 4 



Instructions: Read across the row corresponding to o district listed to the left of the chart. Match the shading intensity to the I 
key below to determine whether the overage reading scale score of this district wos found to be higher than, not sionificontly 
different from, or lower than the district in the column heoding. For example, in the row for Atlanta: Atlanta's score was lower than 
that of the nation, the central dty sample, and New York Qty and Houston, not significantly different from Chicago and Los 
Angeles, and was higher than the District of Columbia. 
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New York City 
Houston 
Atlanta 
Chicago 
Los Angeles 
District of Columbia 




^ ' Disfricf hod higher 
1 overage scoie score 
rhon the disfricf 
lisied of ihe top of 
the figure. 

□ No sigr^ificont 

difference detected 
from the district 
listed ot the top of 
the figure. 

District hod lower 
BJI overage scoie score 
thon the district 
listed ot the top of 
the figure. 



t /^though deemed suffident for reporting, tfie target response rote specified in tb guidelines was not met. 

^ For comparison, ot fourth grode iS percem of students in central city puUk uhrals ond 40 p^cent in puUa schoois notionoDy wae non-Viltim. Ako, &] percent of students m cattrd city pubhc sdtools and 43 percent 
in public schools notiondiy were eltgdile b Iree/rerkicediirke schod lunch. 

NOTL’ Ihe between-dstrict compfflisruts tdce into account sompbng ond measurement errm ond that eoch dsOkt is bang compmed wid every oth^ dstrkt shown. Signifkance is detenmiffid by on oppitaito of o 
mdtipie^tm^jcnson proodure. 

SO URCE: Ui. Deportment of biucohon, Institute of Education Sciemes, Notkmd Centff for Bhicatmn Statistics, Notionol Assessor of Educoirnd Progress (NAEP), 2(X)2 Trkd Urbtm Distnct Reodng As»ssment. 
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At grade 8, the average score for stu- 
dents in the nation as a whole and in 
central cities was higher than the average 
score in any of the districts. The average 
scores for students in Chicago vind Houston 
were higher tlian those in the other districts 



and were not found to differ significantly 
from one iinother. Finally, grade 8 students 
in the District of Columbia outscored those 
in Los .'\ngeles ;ind Atl;mta. Other apparent 
differences were not statistically significant. 



Figure 2.2 Cross’dlstrict comporlsons of overoge reading scale scores, grade 8 public schools: By urbon district, 2002 



Grade 8 



instructions; Read across the row corresponding to a district listed to the left of the chart. Match the shading intensity to the 
key below to determine whether the overage reading scale score of this district was found to be higher than, not signiticantly 
different from, or lower than the district in the column heading. For exomple, in the row for the District of Columbia: The District 
of Columbia's score was lower than the nation, the central city sample, and Chicago and Houston, and higher than Los Angeles 
and Atlanta. 
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0 District hod higher 
overoge scole score 
thon the district 
listed ot the top of 
the figure. 

□ No significant 

difference delected 
from ihe district 
listed ot the top of 
the figure. 

H District hod lower 
overage scole score 
thon the district 
listed ot the top of 
the figure. 



* For comparison, ot dgiiffi grmk 61 percent of students in cenfrol dty pubfic sdiook and 36 percent in public schools notionoHy were non-White. Also, 47 penent of students in central dty pubbc schools and 34 percent 
91 pubbc schools notion^ were ehgible hir free/reduced-prke school hmdi. 

NOIL between-cbstrkt omqnrisons take into aaouitsamp^ and measurenttut error and dmteadi cfislrkt isbemg con^rard with every other (hstrict shown. Si^ifrconce is (ktenrnied by cbi qip&ntion of a 

moltk]b<omp(nson proochira 

SOURCE: Ui. Deportment of friuta^, Institute of Education Scimes, Natkmol Goiter for Educotkm StotsHcs, Nationoi Assessment of Educotiond Progress (NAEP), 2002 Tfd UrbtKi ^strict Asessment. 
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Reading Achievement Level 
Results 

In addition to repotting average reading 
scale scores, N^\EP reports reading perfor- 
mance in terms of percentages of students 
reaching achievement levels. The reading 
achievement levels are Basic, Prvjicient, and 
Advanced. The setting of achievement 
levels is discussed in chapter 1. 

Table 2,2 presents the percentages of 
students at grades 4 and 8 who performed 
below the Basic level, at or above the Bade 
level, at or above the Projicient level, and at 
the Advanced level. At grade 4, the percent- 
age of students performing at or above 
P?v/icient rmged from 10 percent in the 
District of Columbia to 19 percent in New 
York City. In contrast, 30 percent of the 
public-school students across the nation 
performed at or above the Proficient level, 
while 21 percent of all central city public- 
school students reached this level of 



achievement. The percentages of fourth- 
grade students reaching at least the Basic 
achievement level ranged from 31 percent 
in the District of Columbia to 48 percent 
in Houston. In the nation as a whole, 62 
percent of public-school students and 51 
percent of centoil-city students performed 
at or above the Bade level. 

At grade 8, the percentages of students 
performing at or above the Projicient level 
ranged from a low of 8 percent in Atlanta 
to a high of 17 percent in Houston, In 
contrast, 31 percent of students in public 
schools in the nation and 23 percent of 
students in central-city schools performed 
at or above this level. The percenmges of 
eighth-grade students performing at or 
above Basic ranged from 42 percent in 
Atlanta to 62 percent in Chicago, Across 
the nation, 74 percent of public-school 
students and 64 percent of public-school 
students in central cities performed at or 
above the Basic level. 
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Table 2-2 Percentage of students at or above each reading achievement levels grades 4 and 8 public schools: 
By urban district, 2002 





Below Bask 


At or above 
Bask 


At or above 
ProHileat 


At Advanced 


Grade 4 




Notion (Public) 


38 


62 


30 


6 


Control city public ^ 


49^ 


51 *" 


21"" 


4*" 


^Atlanta 


65 


35 *'** 


12"** 


3*'** 


efiksgo 


66 


34 *'** 


11"'** 


2 +,** 


District of Columbia 


69 


31 *^** 


10"'** 


2*'** 


Houston 


52*" 


48*" 


18*" 


3*" 


ios Angeles 


67 *'** 


33 *'** 


11"** 


2 *'** 


New York City ^ 


53*" 


47*" 


19*" 


5 












Notion (Public) 


26 


74 


31 


2 


Centrol dty public ^ 


36*" 


64"* 


23*" 


2"* 


Atlonta 


58 


42 *'^* 


8 




Cfikogo 


38*" 


62** 


15"'** 


1 


District of Columbia 


52 *'** 


48 *'** 


10*'** 




Houston 


41 *'** 


59 V* 






los Angeles 


56 *'^* 


44 


10"'** 





#Percentags rounds to ZHO. 

^ /^though deemed sufBdent for reporting, the target response rota spedhed in the NAEF guideiines was irot met. 

Signifirontiy different hom central dty public sdioois. 

** Significantly different from notion {public schools). 

^ For comparison, at fourth grde 65 percent of students in ranN city pubbc schools and ^0 percent in pid)Sc schools notlonolly w^e non-White. Ako, 61 percent of students in central dty public schools ond 43 
percent in public schmds nolionally were elig^le hv ffee/ reduced-price school hmdi. 

^ For comparison, at u^th grde 61 percent of students in central city public schools and 36 purest in pubhc schools nationally were non-White. Also, 47 percent of studatts in central dty puidic schools and 34 
percent in puhbe schools notiondly wore eiigibie hv bee/ reduced-price school lunch. 

NOIF: Percentages bdow od at or above ^os^moy not add to 1 00, due to rounding. 

SOURCE U.S. Deportment of Education, Institute of Efucotion Sciences, Notool Center for Edcotim) Stotislics, Notionol Assesmientof Edcotiimd Progress fUAEP], 2002 Tried Uihan Distil Reading Amassment. 
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Comparisons Among Districts 
by Achievement Levels 

Figures 2.3 and 2.4 are '‘mileage charts” 
similar to figures 2.1 and 2.2; in this case 
the data comp;ired represent percentages 
of students performing at or above VroficienL 
At grade 4, the percentage of students 
performing at or above Vrofuient in the 
public schools nationiilly was higher than 
the percentages in each of the urban 



districts. The percentage of students 
performing at or above Proficient in central 
city public schools was not found to be 
different from the percentages in Houston 
and New York City, but W'as higher than 
the other four districts. The percentages of 
fourth-grade students performing at or 
above Proficient wzs not found to differ 
significantly across districts. 



fioure 2.3 Cross-district comparisons of percentage of students ot or obove Profideat In reeding, grade 4 public schools: 
By urbon district, 2002 



Grade 4 



Instructions: Read across the row corresponding to a district listed to the left of the chart. Match the shading intensity to the 
key below to determine whether the percentage of students at or above Proficient in this district was found to be higher than, 
not significantly different from, or lower than the district in the column heading. For example, in the row for Chicago: Ihe 
percentage of students at or above Proficient In Chicago was lower than the nation and the central dty sample, and not 
significantly different from any of the other districts. 
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New York City ^ 
Houston 
Atlanta 
Los Angeles 
Chicago 
District of Columbia 




District had higher 
^ percentage than the 
district listed ot the 
top of the figure. 

□ No significant 

difference detected 
from the district 
listed at the top of 
ihe figure. 

O District had lower 
percentage than the 
district listed at the 
top of the figure. 



^ Although deemed sufBdatt for reporting, the target response rate spedfied in the HAS’ guide&ies vras not met. 

^ For comparison, at burth grde 65 p^cent of students in centroi city public schools emd 40 penent in pubfic sdtools nattonofiy were non-White. Also, 6] percent of students in central dty puhBc school ond 43 
patent in public sdtook notiondly were d^ible forfree/reduced-price school lunch. 

NOTE llw betw^n-cfetiid comparisons take imo QcciHjnt SQinp^ and meosurernenl ^r ond «Kh dkirtet b bmg coinpvd with evay odier dls!^ 

ntabpb-ota^arfion procedure. 

SOURCE: U.S. Deportment of Educotion, instHule of Eckjcoiion Sciences, Notiond Center for Education Statistics, llotjonal Assessmoit of Eduto^onai Prioress (NAEP), 2002 Tnd Urbrm Dklrict Renting Assessntant. 
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At grade 8, the percentages of students 
performing at or above Proficient in tlie 
national public schools ;md in central city 
public schools were higher than the per- 
centages in each of the piorticipating urbiin 
districts. Houston had higher percentages 
of students performing at or above Profi- 
cient thixn Atlanta, the District of Columbia, 
and Los ^\ngeles. 

Wdule they 2 ire not displayed in figures, 
tliere were also interesting patterns in the 
percentages of students performing at or 
above Basic. At grade 4, students in piortici- 
pating urban districts were less likely than 
students in public schools in the nation as a 
whole to perform at Basic or higher. How- 
ever, only four of the urban districts — 
Atlanta, Chicago, the District of Columbia, 
and Los ^Vngeles — ^had lower percentages 
of students performing at or above Basic 
thiin central-city schools across the nation. 



Houston and New York had higher per- 
centages of students performing at or 
above Basic than the other four districts. 

At grade 8, a higher percentage of public 
school students across the nation per- 
formed at or above Basic than those in the 
participating urban districts. A higher 
percentage of students in central city 
schools across the nation performed at or 
above Basic thaw did students in Houston, 
Los ^riigeles, the District of Columbia, and 
Atlanta. Students in Chicago and Houston 
were more likely than those in the District 
of Columbia, Atlanta, and Los Angeles to 
perform at or above Basic. Finally, students 
in the District of Columbia were more 
likely to perform at or above Basic tlian 
were those in Atlanta and Los Angeles. 
Other apparent differences were not found 
to be statistically significant. 
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Figure 2.4 Cross-district comparisons of percentoge of students at or obove Profkient in reoding, grade 8 public schools: 
By urbon district^ 2002 



0rode 8 



instructions; Read across the row corresponding to a district listed to the left of the chart. Match the shading intensity to the 
key below to determine whether the percentage of students at or above Proficient \n this district was found to be higher than, 
not significantly different from, or lower than the district in the column heading. For example, in the row for Houston: The 
percentage of students at or above Proficient \n Houston was lower than the national and central city samples, not significantly 
different from Chicago, and higher than Los Angeles, the District of Columbia, and Atlanta. 



Houston 
Chicago 
Los Angeles 
District of Columbia 
Atlanta 




DisIricJ had higher 
^ percentage than the 
district fisted at the 
top of the figure. 

□ No significant 

difference detected 
from the district 
listed ot the top of 
the figure. 

H District hod lower 

percentage than the 
district listed ot the 
top of the figure. 



^ For (ompvison, ot eighth grade 61 percent of studoits m cen^ city public schoob and 36 parent in pubbe uhods notionaily were fl&o, M percent of students in central city public seboob and ZA 

percent in public seboob notionolly were eligibiB for free/ reduced-price school Irnidt. 

NOTE: The between-dhtrict comparisons take into oaount sompbng and meosurement error end thot each dtsfrici is bemg compared wHh ev»y other (bstrict shovm. Sgnifiaince is determined by on cg^dicotion of a 
nwhi(^<on^nrison procedure. 

SOURCE: U.S. Department of bbicolion, Institute of Education Sciences, Notional Center for biucettion Statistics, National Asses^nent of biucaticHia! Progress (NAEP), 2002 Trial Urbem District Reoebng Assessment. 
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3 

Results for Subgroups 

In addition to reporting on die performance of aU students, 
NAEP provides results for a variety of subgroups of 
students for each grade level assessed. The subgroup results 
show how these groups of students performed in comparison 
with one another. X^'Tien additional years of urban district 
assessments are completed, these results will provide a basis 
for examining each subgroup’s progress over time. 

This chapter includes the percentage of students, average 
reading scores, and achievement level results at grades 4 and 
8 for subgroups of students in die five urban districts that 
participated in the Trial Urban District Assessment, for the 
District of Columbia, and for public schools in the nation 
and central cities. Results are reported by gender, race/ 
ethnicity, students’ eligibility for free/reduced-price lunch, 
and parents’ highest level of education. 

Throughout this chapter, the results for students in 
different subgroups within a district are compared to each 
other. Student subgroup results in each district are also 
compared to die central city public-school results. In most 
cases, the average scores or achievement level results for 
central city schools were below those for die national public 
schools. AU differences noted as such in this chapter are 
statistically significant. In interpreting the results, it is 
important to bear in mind that the estimated average score 
for a particular group does not include the whole range of 
performance within tiiat group. Differences in subgroup 
performance canncit be ascribed solely to students’ 
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membership in an identified subgroup. 
Average student performance is affected by 
the interaction of a complex set of factors 
not discussed in this report or addressed by 
N^AEP assessments. 

Performance of Selected 
Subgroups in the 
Urban Districts 

Gender 

Educators and government agencies have 
produced a body of research that is rich in 
data documenting gender differences in 
language arts achievement.^ National results 
for tlie NE\EP 1998 and 2002 reading 
assessments indicated that female students 
outperformed male students at grades 4, 8, 
and 12. 

Table 3.1 shows average reading scores 
and achievement level results by gender at 
grade 4. For five of the sLx urban districts at 
grade 4, female students had higher average 
scores than male students. The apparent 
four point difference between average 
scores for male and female students in 
Houston was not found to be statistically 



significant. Average scores for male students 
at grade 4 ranged from 185 in tlae District 
of Columbia to 204 in Houston. The 
central city public school average score for 
male students was 204. Average scores for 
male fourth-grade students in Houston and 
New York City were not found to differ 
significantly from the average score for 
central city public schools, but male stu- 
dents in Atlanta, Chicago, the District of 
Columbia, and Los Angeles scored lower 
than the average for central city public 
schools. 

Average scores for female students at 
grade 4 ranged from 194 in Los ^'Vngeles to 
213 in New York City. The central city 
public school average score for female 
fourth -graders was 211. Average scores for 
female tourth -grade students in Houston 
and New York City were not tound to differ 
significantly from the average score for 
centr;il city public schools, while the average 
score for female students in each of tlae 
other districts was lower than the average 
for central city schools. 



^ US. Deparimeut of Education. (2001). RsatUngjor UndersUmding: Tonmds an R c'~ D Program in Poiading Comprehension. 
Washington, DC; Aiitlioi:. 

MacMillan, P. (2000). Sijmiksineous Measurement of Reading CTro\xd:h, Gender, and Relative- Age Effects: Many 
Faceted Rasch Applied to CBM Reading Scores. Journal of Applied Measurement / (4), 393—408. 
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Table 3.1 Average reading scale scores and percentage of students at or above each achievement level, by gender, 
grade 4 public schools: By urban district, 2002 





' Percentage 
of students 


Average 
scole score 


Below Bask 


At or above 
Bask 


At or above 
Profitieaf 


Advanted 


Mule 


Notion (Public) 


51 


214 


41 


59 


26 


5 


Central city (Public) ’ 


50 


204"" 


53"" 


47 "" 


19"" 


3*» 


Atlanta 


47 


191 


69 *'*" 


31 *'"" 


11 *'"" 


2 ** 


Chicago 


50 


189 


70 *'"" 


30 *'*" 




1 *- 


District af Columbia 


49 


185*'"" 


74 


26 


8 *'*" 


1 


Houston 


51 


204"" 


55 *" 


45 "" 


16"* 


3 


Los Angeles 


51 


188"'*" 


70 "'*" 


30 "'*" 


10 "'*" 


1 


New York City ^ 


50 


199 "* 


61 


39 *'"" 


14** 


3*" 


Fettiale 


Nation (Public) 


49 


220 


35 


65 


33 


8 


Central city (Public) ^ 


50 


211 "" 


45 *" 


55 *" 


24 "" 


5*" 


Atlanta 


53 


200 "'"" 


60 "'** 


40 "'*" 


13"'** 


4 .. 


Chicago 


50 


198"'"" 


62 *'"" 


38 


12 *'"" 


2 ..’F 


District of Columbia 


51 


196*'"" 


64 *'"" 


36 


11 *'"" 




Houston 


49 


208*" 


50*" 


50 "" 


19*" 


3** 


Los Angeles 


49 


194"'"" 


64"'"" 


36 *'*" 




2 ... 


New York City ^ 


50 


213"" 


45 *" 


55 "" 


23 "" 


7 



^ Althoush deemed suffident for reporting, the target respome rote specified in the NAEP gmdelines was not met. 

^ For comporison, ot fourth grode 65 percent of students in centrol city public schools ond 40 percent in public schools notionoHy were non-White. Also, 61 percent of students in central cHy public schools 
and 43 percent in public schools notionoily were eligible for free/reduced-price school lunch. 

* Signihcontiy different from centrol city public schools. 

SIgnifkontly different from notion (public schools). 

NOH: Percentages below ond ot or above Basic may not odd to 1 00, due to rounding. 

SOURCE: U.S. Deportment of Educotion, institute of Educotion Sciences, Notional Center for Educotion StatisHcs, Notionol Assessment of EducoHonol Progress (NAEP), 2002 Triol Urbon District Reeding 
Assessment. 
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Table 3.2 displays average scores and 
achievement level results by gender for 
grade 8. In every district female students 
scored higher, on average, tlum male stu- 
dents. Average scores for male eighth-grade 
students ranged from 231 in Atlanta to 245 
in Chicago. The average score for eighth- 
grade male students in central city schools 
was 250, which was liigher than in each of 
the five urban districts. 



For eighth-gride female students, average 
scores ranged from 240 in Atlanta to 254 in 
Chicago. The average score for female 
eighth-grade students in central city public 
schools was 258, which was higher than the 
average score in every district except Chicago. 



Table 3.2 Average reading scale scores and percentoge of students at or above each achievement level, by gender, 
grade 8 public schools: By urban district, 2002 





Percentage 
of sfiidetits 


Average 
scale score 


Below Bask 


At or above 
Bask 


At or obove 
Profideat 


Advanced 




Notion (Public) 


50 


258 


30 


70 


26 


2 


Central city (Public) ^ 


50 


250 


40 


60 ** 


19** 


1 


Atlanta 


49 


231 


63 


37*'** 


6*'*’ 


# 


Chicago 


50 


245 


43 ** 


57** 


12*'** 


1 


District of Columbia 


47 


235 


58*’** 


42 


9 


# 


Houston 


51 


243 


47 *'** 


53*'** 


13*’** 


# 


Los Angeles 


53 


233 


61 *'** 


39 *'** 




# 


emaie 


Notion (Public) 


50 


267 


21 


79 


36 


3 


Central city (Public) ^ 


50 


258 


31 ** 


69** 


26 *’ 


2** 


Atlanta 


51 


240 


53 *'** 


47 




# 


Chicago 


50 


254 "" 


33 ** 


67 ** 


17 *'*’ 


1 


District of Columbia 


53 


245 


46*'*’ 


54*'*’ 


] ] 


1 


Houston 


49 


253 


35 ** 


65** 


21 


1** 


Los Angeles 


47 


241 


51 *'** 


49 


^ 2 


j 



# Percentage rounds to :^ro. 

^ For comparison, at ei^ grade 61 percent of students m central city public schools and 36 percent in public schools nationaBy were non-White. Also, 47 percent of students in central city public schools 
and 34 percent in public schools notiondly were digiblefor free/reduced-prica school lunch. 

* Significontly different from central city public schools. 

Signihccmtly different from nation (puldic sdroob). 

KOTl: Percentages below and at or above Basi( moy not add to 1 00, due to rounding. 

SOURCE: Ui. Department of Blutation, Institute of Bhicotion Sciences, NaHonol Center for Bfucatian Stotistks, Notionol Assessment of Echjcational Progress (NAEP), 2002 Tritd Urban District Reading 
Assessment. 
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An additional way to compare student 
performance is to focus on the size of the 
difference in scores or “gap'' between the 
subgroup average scores. Figure 3.1 presents 
these ‘‘gaps" in average reading scores 
between female and male students in each 
district as well as in the national and central 
city public schools. At grade 4, the gap 
between the average scores of temale and 



male students in the District of Columbia 
was wider than the gap in public schools in 
the nation. The average-score gaps between 
male and female students in all the other 
participating urban districts at both grades 4 
and 8 were not found to differ significantly 
from the average-score gaps in public 
schools in centred cities or the nation. 



Figure 3.1 Gaps in average reading scale scores, by gender, grades 4 and 8: By urban district, 2002 



Female average score minus Mole average score 

Grade 4 



Notion (Public) 
Centrof city (Public) 
Atlonto 
Chkogo 
District of Columbio 
Houston 
Los Angeles 
New York City 






10 ^ 



14 



-10 0 10 20 30 40 50 

Score gaps 



60 



Female overage score minus Male overoge score 

Grade 8 



Nation (Public) 
Control city (Public) 
Atlanta 
Chicago 
District of Columbia 
Houston 
Los Angeles 




^ Although deemed sufficient br reporting, the target response rote specified in the NAEP guidelines was not met. 

^ For comparison, at fourth grade 65 percent of students in cenkoi city public schools and 40 percent in poblic schools notionolly were non-VIfltife. Also, 61 percent of stud»its In central city public schools 
end 43 percent in public schools notiono&y were eligdde br free/reduced-price school lunch. 

^ For comparison, ot eighth grade 6) percent of students in cenkol dty public schools ond 36 percent in puUk schools notionolly were non-V4ute. Also, 47 percent of students in central city public schools 
and 34 percent in public schools [rationally were eligiUe for free/reduced-price school lunch. 

Signiftcontiy different from notion (public schools). 

NOTE: Score gaps ore colcuktrad hosed on differences between unrounded overage stole scores. 

SOURCE: U.S. Deportment of Educotion, Institute of Education Sciences, Notionoi Onler for Educotion Stotlstics, Notionoi Assessment of Educotiond Progress (NAEP), 2002 Reoding Assessment. 
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Race/Ethnicity 

For the purpose of studying the progress of 
subgroups, N^\EP collects information 
from school records on the racial/ edinic 
identification that best describes each 
participating student. The six mutually 
exclusive categories are White, Black, 
Hispanic, Asian /Pacific Islander, American 
Indian (including ^\laska Native), and Other. 
For fiirther details, see '‘NiVEP Reporting 
Groups” in appendix C. 

Tables 3.3 and 3.4 show average scores 
and achievement level results by racial/ 
ethnic group membership for public-school 
students in the urban districts at grades 4 
and 8, respectively. Only the race/ethnicity 
categories with sufficient membership to 
meet reporting requirements in the urban 
districts are reported below. 

The distribution of students in terms of 
race/ethnicity in the urban districts differs 
from that of the national distribution in 
public schools. Whereas White students 
comprise 60 percent of the national public 
sample at grade 4 and 64 percent at grade 8, 
in the urban districts shown here White 
students make up a mviximum of 15 percent 
of the district samples at grade 4 (New York 
City) and 11 percent at grade 8 (Chicago). 
Black and/or Hispanic students constitute 
the majority in each of the urban districts in 
the trial assessment. Hispanic students made 
up half or more of die sample in Houston 
and Los Angeles at both grades 4 and 8. 
Black fourth- and eighth -grade students 
made up more than 80 percent of the 
sample in botJi Atlanta -and die District of 
Columbia. 



In five of the six of urban districts in 
which a reliable comp;irison could be made, 
White fourth-graders had higher average 
scores than their Black and Hisp;inic peers. 
In Chicago, Hispanic fourth-graders had 
higher average scores di:in Black fourth- 
graders. Any apparent ditference between 
Hispanic and Black students' average scores 
in the District of Columbia, Houston, Los 
/higeles, and New York City was not found 
to be statistically significant. The White/ 
Hispanic and the Hispanic /Black compari- 
sons in Adanta could not be tested for 
statistical significance due to insufficient 
sample size. 

At grade 4, Black students in Chicago, die 
District of Columbia, ;ind Los .^\ngeles 
scored lower th^in their counterparts in 
central city public schools. Hispanic stu- 
dents in Chicvigo, Houston, and Los ^Ynge- 
les also scored lower, on average, than their 
counteqiarts in central city public schools. 
White students in Atlanta and the District 
of Columbia scored higher than their 
counterparts in central city public schools. 
Average scores for WTiite students in Chi- 
cago, Houston, Los Angeles, and New York 
City were not found to differ significantly 
from those of their counterparts in central 
city public schools. Asian/Pacific Islander 
fourth-graders in New York City^ had higher 
average scores than their counterparts in 
central city public schools. The average 
score for Asivin/Pacific Islander students in 
Los Angeles was not found to differ signifi- 
cantly from that of Asian /Pacific Isliinder 
students in central city schools. For the 
other districts, the samples of Asian/Pacific 
Islander fourth -graders were insutficient tor 
reliable significance testing. 
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Toble 3.3 Average reading stale stores and pertentage of students at or above eath othievement level, 
by rote/ethnitity, grade 4 publit sthools: By urban distritt, 2002 



Grade 4 





Percentage 


Average 




At or above 


At or above 






of students 


scale score 


Below Bask 


Bask 


Profitieaf 


Advamed 


White 














Nation (Public) 


60 


227 


26 


74 


39 


9 


Central city (Public) ^ 


35 


225 


29 


71 ** 


37 ** 


9 


Atlanta 


6 


250 


14 


86 *'** 


67*'** 


34*'** 


Chicago 


10 


221 


36 


64 *'** 


35 


9 


District of Columbia 


3 


248 * 






66*'** 


28 


Houston 


10 


233 


21 


79 


45 


13 


Los Angeles 


9 


223 


30 


70 


38 


9 


New York Gty ^ 


15 


226 


29 


71 


35 


10 


Block 














Notion (Public) 


18 


198 


61 


39 


12 


1 


Central dty (Public) ^ 


31 


194 


65 ** 


35 ** 


10** 


1 


Atlanta 


90 


192 "" 


68 ** 


32 ** 


8** 


1 


Chicago 


48 


185 


75 


25 *'*’ 


5V’ 


# 


District of Columbia 


88 


188 


72 


28 *'** 


7*'** 




Houston 


37 


200 


60 


40 


12 


1 


Los Angeles 


12 


186 


75 V* 


25 *'** 


6** 


# 


New York Gty ^ 


36 


197 


63 


37 


9 


2 


Hispanic 














Notion (Public) 


17 


199 


57 


43 


14 


2 


Central city (Public) ^ 


27 


199 


59 


41 


13 


2 


Atlanta 


3 








♦♦♦ 




Chicago 


37 


193 


67 *'** 


33 *'** 




1 


District of Columbia 


7 


193 


66 


34 


8** 


1 


Houston 


50 


203 * 


55 


45 


14 


2 


Los Angeles 


72 


185 


74 


26 *'** 


■j 


1 


New York City ^ 


40 


201 


58 


42 


15 


3 


Asian/Pacific Islonder 














Notion (Public) 


4 


223 


31 


69 


36 


9 


Central city (Public) * 


5 


217** 


38 ** 


62 ** 


29 ** 


7** 


Atlanta 


# 












Chicago 


3 












District of Columbia 


1 








♦♦♦ 




Houston 


3 










♦*+ 


Los Angeles 


6 


218 


30* 


70* 


26 


3 


New York City ^ 


8 


235 * 


22* 


78* 


50* 


20* 


American litdion/Alaska Notive 














Notion (Public) 


1 


207 


49 


51 


22 


5 


Central city (Public) ^ 


1 


207 


50 


50 


23 


6 


Atlanta 


# 












Chicago 


1 












District of Columbia 


# 






♦+* 






Houston 


# 




*** 








Los Angeles 


1 


vNcvc 










New York Gty ^ 


# 













O 

ERIC 



# Percentoge rounds (o zero. 

Although deemed suffkient for reporting, the tmget response rote ^eclBed in the NAEP guideimes wos not met. 

Signjfkontly different from tentro) dty pufalk schools. 

* Slgnifkontly different from notion (public schools). 

“ Sompie size h Insufficient to permit a reliahle estimate. 

For comparison, ot fourth grade 65 percent of students in control dty public schools and 40 percent in public schools notionoSy were non V^te. Also, 61 percent of students in central dty public schools 
ond 43 percent in public schools nohonaify were eligible for free/reducedipnce school lunch. 

NOTE: Percenioges below and ot a above Basit may not odd to 100, due to rounding. 

SOURCE: U.S. Department of Education, Institute of Education Sciences, Notional Cent^ for Educotion Statistics, Notionol Assessment of Educohonal Progress (Ni^P), 2002 Trial Urlmn District Reodmg 
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Table 3.4 displays the performance results 
by race/ethnicity for grade 8. White eighth- 
graders outperformed Black eighth-graders, 
on average, in AtJiinta, Houston, and Los 
Angeles. The apparent difference in Chi- 
cago was not found to be statistically 
significant, rind the sample size in the 
District of Columbia was insufficient to 
permit a reliable comp'ririson. White eighth- 
graders dso had higher average scores than 
their Hispanic peers in Houston and Los 
iVngeles. The apparent difference in Chi- 
cago was not found to be statistically 
significant, and the sample sizes in Atlanta 
and the District of Columbia were insuffi- 
cient to permit a reliable comp;irison. Black 
eighth -graders in Los Angeles outperformed 
their Hispanic peers, on average. The 
apparent difference in Chicago, the District 
of Columbia, and Houston between these 
two groups of students was not statistically 
significant, and the sample size was 
insufficient in Atlanta to permit a reliable 
comparison. 



Black eighth-graders in Atlanta, tlie 
District of Columbia, ;ind Los iingeles had 
lower average scores thaia their counterparts 
in central city public schools. Los iVngeles 
was the only district where Hispanic eighth- 
graders scored lower, on average, than 
Hispanic students in central city public 
schools. The other app;irent differences 
between districts iond central city public 
schools for Black and Hispanic students’ 
average scores were not found to be statisti- 
cally significant. The average score for 
White students in Houston was higher than 
the average for White students in central 
city' public schools, while all other districts 
were not found to differ significantly from 
the central city^ public schools. Average 
scores for Asian/Pacific Isliinder students in 
Los Angeles were not found to differ 
significantly from the scores of their coun- 
terparts in central city schools. The sample 
of Asian/Pacihc Islander students in each 
of the other districts was too small to 
support significance testing. 
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Tobie 3.4 Average reading scale scores and percentage of students at or above each achievement level, 
by race/ethnicity, grade 8 public schools; By urban district, 2002 



Grades 



urooeo g 


Percentage 


Average 




At or above 


At or above 






of sfiidents 


scale score 


Below Bask 


Bask 


Profitieaf 


Advanced 


White 














Nation (Public) 


64 


271 


17 


83 


39 


3 


Central city (Public)^ 


39 


269 


20** 


80* *" 


37 


3 


Atlanta 


5 


275 


16 


84 


47 


5 


Chicago 


n 


266 


25 


75 


31 


5 


District of Columbia 


3 












Houston 


8 


279*'** 


13 


87 


47 


5 


Los Angeles 


10 


264 ** 


27 ** 


73 ** 


33 


3 


Black 














Nation (Public) 


15 


244 


46 


54 


13 


# 


Central dty (Public) ^ 


28 


241 ** 


50 ** 


50** 


11** 


# 


Atlanta 


92 


233 *'** 


61 *'** 


39 *'** 




# 


Chicago 


50 


245 


43* 


57* 


10 


# 


District af Columbia 


88 


238 *'** 


54 *'** 


46 *'*" 


8 V’ 


# 


Houston 


31 


247 


40 


60 


15 


# 


Las Angeles 


14 


236 "'*" 


57 ** 


43** 


8 


# 


Hispanic 














Nation (Public) 


15 


245 


44 


56 


14 


# 


Central dty (Public) ’ 


25 


244 


46 


54 


13 


# 


Atlanta 


2 












Chicago 


35 


248 


39 


61 


12 


# 


District of Columbia 


7 


240 


47 


53 


11 


# 


Houston 


58 


243 


48 


52 


13 


# 


Los Angeles 


67 


230 *'** 


64 *'** 


36*'*" 


5 *'** 


# 


Asion/Pocific Islander 














Nation (Public) 


4 


265 


25 


75 


34 


3 


Central dty (Public) ^ 


6 


258** 


32 ** 


68** 


27 *" 


1 ** 


Atlanta 


1 












Chicago 


2 










A** 


District of Columbia 


2 










♦ AA 


Houston 


3 




*♦* 




-r+* 




Los Angeles 


9 


259 


27 


73 


26 


1 


Americon Imlion/Aioska Native 














Natian (Public) 


1 


252 


36 


64 


18 


1 


Central dty (Public) ^ 


1 


258 


28 


72 


21 


2 


Atlanta 


# 










*** 


Chicago 


1 








*** 




District of Columbia 


# 


V** 










Houston 


# 










*V* 


Los Angeles 


# 













# Percentoge rounds io zero. 

* Signifkontty dif^rent from rentroi city public schools. 

Signifkontty different from notion (public schools). 

Sample size is insufficient to permit o reliable estimate. 

^ For comparison, ot eighdi grode 61 percent of stud^ts m centroi dty public schools ond 36 percent in public schools notionally wore non-White. Also, 47 penent of studoits in centrd city puUic schools 
and 34 percent in pubb'c schools notionally were eligible for free/reduced-price schod lunch. 

NOTE: Percentoges below ond ot or above Basic may not odd to 100, due m rounding. 

SOURCE: U.S. Deportment of Education, Institute of Education Sciences, Notionol Center for Education Stothtks, Notionol Assessmoit of Educotionol Progress (NAEP), 2002 Trial Urban District Rending 
Assessment. 
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Average scale score differences or ‘'gaps'’ 
between White and Black students and 
between White and Hispanic students ;ire 
presented in figure 3.2. At gnide 4, the gaps 
between the average scores for White and 
Black students in Atlanta and the District of 
Columbia were wider than the correspond- 
ing gaps between White and Black students 
in public schools in the nation and in central 
cities. The average-score gaps betV'Cen 
White and Hispanic fourth-graders in the 
District of Columbia and Los ihigeles were 



wider than the corresponding gaps in the 
nation iind in central cities. 

At grade 8, the average-score gap be- 
tween White and Black students in Atlanta 
was wider than the corresponding gaps in 
public schools in the nation and in central 
cities. The average-score gaps between 
White and Hispanic eighth-graders in 
Houston and Los Angeles were wider than 
die corresponding gaps in the nvition and in 
central cities. 



Figure 3.2 Gaps In average reading scale scores, by race/ethnicity, grades 4 and 8: By urban district, 2002 



White average score minus Black average score 

Grade 4 




Score gaps 

White average score minus Hisponic average score 

Grade 4 



Notion (Pubik) 
Central city (Public) ^ 
Atlanta ^ 
Chicago 
Disirief of Columbia 
Houston 
Los Angeles 
New York Gly * 


m 98 


m Ifi 


m 

m 98 


• zo 

■ SS*'** 


• JJ 

m 9P 


• LJ 

m 98*'*^ 


w 00 

A 9*5 


• Zj 


-10 0 10 20 30 40 50 60 

Score gaps 



White average score minus Black average score 

Grade 8 




Score gaps 

White average score minus Hispanic average score 

Grade 8 




^ Although deem&d suffkisnt for reporting, the target response rote seethed in the NAEP guidelines wos not met. 

* Significantly different from centrd city public schools. 

'* Significantly different ^om notion (public schools). 

I For comparison, ot fimrlh grode 65 percent of students in coitrol city public schooh ond 40 percent In public schools notionolly were non-Vl^ite. Also, 61 percoit of students in central city puidk schools 
and 43 percent m public schools notionolly were eligible lor ^ee/reduced-price school kinch. 

^ For comparison, ot eighdi grode 61 percent of students in centrol city public schools and 36 percent in public schools notionolly were non-White. Also, 47 percent of student in central city public schools 
ond 34 |»rcem m pubb'c schools notionolly were eligible for free/ reduced-price school hinch. 

^ Somple sizes were insufficient to permit o rellohle eshmote for fourth-ond eighth-grode Hisponic students in Ationto ond for eighih-grode White students in the DHtrict of ColumiHO. 

NOTE: Score gops ore colculoted based on differences between unrounded overage scale scores. 

SOURCE: U.S. (leportment of Educotitm, Institute of Education Sciences, Notional (enter for &iucation Stohstics, Notional Assessment of Educational Ingress (NAEP), 2002 Reoding Assessmoit. 
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Free/Reduced-Price Lunch 
Eligibility 

N^\EP collects data on a viiriety of factors 
that may affect the context of learning. One 
of these is eligibility for the federal program 
providing free/ reduced-price school 
lunches. The free/reduced-price lunch 
component of the Nation;il School Lunch 
Prognim (NSLP), offered through the US. 
Depiirtment of Agriculture (USDA), is 
designed to ensure that children near or 
below the poverty line receive nourishing 
meals. This program is available to public 
schools, nonprofit private schools, and 
residential childcare institutions. Eligibility is 
determined through the USDiVs Income 
Eligibility Guidelines, and is an indicator of 
low income.“ Tables 3.5 and 3.6 present 
these data for grades 4 and 8. 

At grade 4, the percentages of students 
from the urban districts eligible for free/ 
reduced-price lunch ranged from 72 percent 
in Houston to 88 percent in Chicago. At 
grade 8, the percentages in four of the 
urban districts ranged from 68 percent to 84 
percent. Information on tlie free/ reduced- 
price lunch data for Los jVngeles at grade 8 
is not reported because the data received 
did not meet reporting standards. 



N^\EP national report c^irds across all 
subjects have consistently reported lower 
performance scores for students eligible for 
free/reduced-price lunch than for tliose 
who were not eligible. This pattern was also 
generally the case in the urban district data. 
At grade 4, students not eligible for the 
free/reduced-price lunch had higher average 
scores tlian those eligible in every district. 

Fourth-grade students eligible for free/ 
reduced-price lunch in Athmta, Chicago, the 
District of Columbia, and Los ^\ngeles had 
lower average scores than their eligible 
counterp^irts in central city public schools. 
No signific^mt difference was found be- 
tween the average scores for eligible stu- 
dents in central city public schools and 
those in Houston and New York City. In 
comp:irison with ineligible students in 
central city public schools, ineligible stu- 
dents in Atkinta, die District of Columbia, 
and Los Angeles scored lower. No signifi- 
cant difference was found between the 
central city public schools ;md the other 
districts for students who were not eligible. 



^ US. General Services Adnunistratiou. (2001). Catalog of Federal Domestic AssL^Umce. Washington, DC: Executive 
Office of the President, Office of Management and Budget. 
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Tobie 3«5 Average reading scale scores and percentage of students at or above each achievement level, by eligibility for 
free/reduced-price school lunch, grade 4 pubUc schools: By urban district, 2002 





Percentage 
of sfiidents 


Average 
scale score 


Below Bask 


At or above 

Bask 


At or above 
Profitieaf 


Advamed 


Eligible 


Nation (Public) 


43 


202 


54 


46 


16 


2 


Central city (Public) ^ 


61 


198 "" 


60 "" 


40 "" 


13"" 


2 


Atlanta 


74 


189 


71 *'*" 


29 *,**!' 


J 


1 **’ 


Chicago 


88 


190 


70"'"" 


30"'"" 


8"'"" 




District of Columbia 


78 


185*'"" 


75 


25 




#*'"' 


Houston 


72 


199 


60 


40 


11"* 


1 


Los Angeles 


79 


186 


73 


27 


J ’t: +» 


^ :4c:<ri 


New York Gty^ 


73 


201 


58 


42 


15 


3 


No! eligible 


Notion (Public) 


50 


229 


24 


76 


41 


10 


Central city (Public) ^ 


33 


225 "* 


29 "" 


71 *" 


37 ** 




Atlonto 


16 


214 


45 *'"* 


55 


27"."* 


10 


Chicago 


8 


222 


35 "* 


65 "" 


33 


11 


District of Columbia 


21 


210 


48 


52 


23"'*’ 


7"* 


Houston 


24 


226 


28 


72 


39 


9 


Los Angeles 


5 


199 


58 + 


42 


14*'"" 


1 


New York City * 


16 


219"" 


38"" 


62 "" 


30 


8 


Information not ovatlable ^ 


Notion (Public) 


7 


217 


38 


62 


30 


7 


Central city (Public) ^ 


6 


210"" 


47 "" 


53 "" 


24"* 


5 


Atlanta 


11 


211 


51 


49 


22 


7 


Chicago 


4 


206 


52 "" 


48 "" 


19 


4 


District of Columbia 


1 


**¥ 




*** 


**T 


*¥¥ 


Houston 


4 


+*¥ 




.+*▼ 






Los Angeles 


16 


215 


40 


60 


28 


6 


New York Oty ^ 


11 


221 


38 


62 


28 


11 



# PertentDgd rounds to zero. 

^ /^though deemed suffideni br reirarting, the target response rote spetihed in the MAO’ guidelmes was not met. 

* SigniHcontiy different from tentroi dty public schools. 

Signifkcmtiy different ftom notion (public schoob). 

*** Sample size is insufficient to permit a rdioble estimote. 

^ For comparison, at fourth grade 65 percent of students in centrol city public schoob ond 40 percent in public schoob notionolly were non-White. Abo, 61 percent of shjdents in centrol dty puUic schoob 
ond 43 percem in public schoob notionolly were eligible for bee/reduced-price school lunch. 

^ If school records were not ovoiloble, Ihe student wos clossified os 'Informotion not ovailable." If the school did not portidpate in the progrom, oil students in thot school were dosstfied os "Information 
not ovoiloble." 

NOTH: Percentages below ond at or above Sosit may not odd to 1 00, due to rounding. 

SOURCE: U.S. Deportment of Educotion, Institute of Hducolion Sciences, National Centw for Hducotion Statistics, Notional Assessment of Educational Progress (NAEP), 2002 Trio! Urbon Obfrict Reoding 
Assessment. 
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Table 3.6 shows student performance 
data at grade 8 by free/ reduced-price lunch 
eligibility. Eighth-graders not eligible for 
free/ reduced-price lunch had higher average 
scores in comparison to eligible students in 
every district except Chic;igo, where the 
apparent difference was not statistically 
significant. 



In comparison with central city public 
schools, the average scores for eligible 
students were lower in Adanta and the 
District of Columbia, and were not found 
to differ significantly in Chicago and Hous- 
ton. Among students who were not eligible, 
average scores in Atlanta, the District of 
Columbia, and Houston were lower than in 
central city schools. No significant differ- 
ence was detected ben.veen central city^ 
schools and Chicago in the average scores 
of students who were not eligible for free/ 
reduced-price school lunch. 



Toble 3.6 Average reading scale scores and percentage of students at or above each achievement level, by eligibility (or 
free/reduced-price school lunch, grade 8 public schools: By urban district, 2002 





Percentage 
of students 


Average 
scale score 


Below Bask 


At or above 
Bask 


At or obove 
Profident 


Advanted 


Eligible 


Notion (Public] 


34 


249 


40 


60 


17 


1 


Central dty (Public) ^ 


47 


244 


47 ** 


53** 


12** 


#** 


Atlonta 


76 


233"'"" 


62*'** 


38*'*" 


6*'*" 


# 


Chicago 


84 


246 


41 


59 


11** 


# 


District of Columbio 


68 


235 


57*-** 


43 "■"* 


6*"* 


# 


Houston 


68 


243*" 


48 ** 


52 *" 


13 


# 


No! eligible 


Notion (Public) 


57 


271 


17 


83 


40 


3 


Central dty (Public) ’ 


41 


268 ** 


21 ** 


79 ** 


36 ** 


3 


Atlanta 


20 


244*-** 


47 *'** 


53 *'"* 


12*'** 


1 


Chicago 


10 


267 


24 


76 


36 


4 


District of Columbia 


31 


251 *'** 


39 *■** 


61 "■*" 


18*'** 




Houston 


29 


261 *'*" 


25 


75 


26*'** 


2 


Information not available ^ 


Notion (Public) 


10 


264 


25 


75 


32 


4 


Central dty (Public) ^ 


12 


253 ** 


36 ** 


64** 


21 ** 


2 


Atlonta 


4 












Chicago 


6 


268* 


21* 


79" 


34 


7 


District of Columbia 


1 












Houston 


3 




*** 









# Percenki^B rounds to zero. 

' SigniBcondy different from centra] city {uiblic uhoois. 

** Signifitantly different from notion (public sdtoob). 

^mpie size is insufficient to permit o reliable estimate. 

^ For companson, ot eighth grode 61 percent of student in centrol dty public schools ond 36 percent in public schools notionoOy nan-V^liite. 

^ tf school records were not avoiioble, the student wos dassiBed os Information not ovmloble.'' if the school did not participate in the jvogrom, all students in diet school were dassiBed os "Information 
not ovodolde.'* 

NOTE: Percentages below ond at or above Basic may not odd to 100, due m rounding. 

SOURCE: U.S. De{mrtment of Education, Institute of Bfucotlon Sciences, National ^nt» for Education Stothtics, Nationol Assesanem of Bfucotionol Progress (NAEP), 2002 Tried Urbon District Reoding 
Assessment. 
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Parents' Highest Level 
of Education 

Eighth-grade students who participated in 
the NAEP 2002 national reading assess- 
ment, including those in the Trial Urban 
District Assessment, were asked to indicate 
the highest level of education completed by 
each parent. Five response options were 
offered: did not finish high school, gradu- 
ated from hi^ school, some education after 
high school, graduated from college, or “I 
don’t blow.” The highest level of education 
reported for either parent was used in the 
analysis of this question. The question was 
not posed to fourth-graders. 

Table 3.7 presents tJie data for the urban 
districts at grade 8. NiUI-P assessments 
typically find a positive relationship benveen 
student reported parentil education and 
student achievement: the higher die parental 
education level, the higher the average 
reading score. This pattern was also gener- 
ally the case for the trial urban districts. 

The District of Columbia had the highest 
percentage of eighdi -graders (40 percent) 
who reported that at least one parent had 
graduated from college. However, for all of 
the urban districts, the percentage of 
students who reported that at legist one 



parent graduated from college was lower 
than that of public schools nationally. 
Houston iind Los ^\ngeles had the highest 
percentages of students who reported that 
neither p-arent had finished high school (21 
and 19 percent, respectively). For these two 
districts and Chic^igo, the percentage of 
students who reported that neither p;irent 
finished high school w^s greater than that 
for national public schools. 

The apparent difference in average scores 
between students in central city public 
schools who reported tliat at least one 
parent graduated from college and their 
counterparts in Houston was not found to 
be statistically significant. How^ever, students 
in Atlanta, Chicago, the District of Colum- 
bia, and Los Angeles who reported tliat at 
least one parent graduated from college 
scored low^er, on average, tlimi their counter- 
parts in central city public schools. Students 
in Atlanta and Los ^\ngeles who reported 
that neither parent had finished high school 
had lower scores than their counterparts in 
central city public schools. Students in 
Houston who reported that neidier parent 
graduated from hig^ school scored higher, 
on average, than their counterparts in 
central city public schools. 



^ Cjrigg, W S., Duaiie, M. C, Jiii, Y, and Campbell, J- IC (2003). The NcUton’s Report C^rd: Reading 2002 (NCES 2003— 
521). Wasliiugtoii, DC; U.S. Departuieiit of Education, Institute of Education Sciences, National Center for 
Education Statistics. 
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Table 3.7 Average reading scale scores end percentage of students at or above each achievement level, by student- 
reported parents^ highest level of education, grade 8 public schools: By urban district, 2002 





Percentage 
of students 


Average 
scale score 


Below Bask 


At or above 
Bask 


At or above 
Profitieaf 


Advamed 


Less than high school 


Notion (Public) 


7 


247 


42 


58 


14 


# 


Central dty (Public) ^ 


9 


243 


48 


52 


11 


# 


Atlanta 


7 


233’'*’ 


66 ’*’ 


34 ’'*’ 


8 


# 


Chicago 


14 


246 


43 


57 


10 




District of Columbia 


7 


240” 


46 


54 


6” 


# 


Houston 


21 


251’ 


38 


62 


17 


1 


Los Angeles 


19 


234 *•” 


61 


39 


7” 


# 


Graduated high school 


Notion (Public) 


18 


256 


31 


69 


21 


1 


Central dty (Public) ^ 


18 


248” 


42 ” 


58 ” 


13” 


# 


Atlanta 


26 


233 


63 ’” 


37 


4 *.♦* 


# 


Chicago 


19 


246” 


40 


60 


^ *♦* 


# 


District of Columbia 


21 


235*'” 


57’'” 


43 


5’'” 


# 


Houston 


19 


242 ” 


48” 


52 ” 


9” 


# 


Los Angeles 


14 


233’'” 


61 


39 ' 




# 


Some education after high school 


Notion (Public) 


20 


267 


19 


81 


33 


2 


Central city (Public) ^ 


19 


260” 


27 ” 


73” 


25 ” 


1” 


Atlanta 


22 


241 


50 *'** 


50 *'” 


8*'” 


# 


Chicago 


22 


260” 


24 


76 


20” 


1 


District of Columbia 


18 


247’” 


43 *■** 


57 *■” 


12*” 


# 


Houston 


15 


260 ” 


25 


75 


24 


1 


Los Angeles 


16 


249’'*’ 


40 *'*’ 


60 ’-*’ 


17’*’ 


1 


Groduoted college 


Nation (Public) 


46 


273 


17 


83 


42 


4 


Central dty (Public) ’ 


41 


266 *’ 


25” 


75 ” 


34” 


3 


Atlanta 


35 


243 


49 


51 


13’” 




Chicago 


31 


255 


33 


67 ’*’ 


20*'*’ 


3 


District of Columbia 


40 


247 


45 


55*'” 


15*'” 




Houston 


28 


262” 


26 ” 


74 *’ 


29” 


2 


Los Angeles 


26 


251 *'” 


40 *■” 


60 


21 


1 


Unknown 


Nation (Public) 


9 


246 


44 


56 


14 


# 


Central dty (Public) ^ 


12 


241 ” 


50” 


50 ” 


11” 


# 


Atlanta 


10 


229 **” 


67 ’” 


33’” 




# 


Chicago 


15 


242 


48 


52 


11 


# 


District of Columbia 


14 


231 ’*’ 


65 *'” 


35 




# 


Houston 


17 


235 ’* 


57” 


43 ” 


7 


# 


Los Angeles 


26 


228 *'” 


67 


33 




# 



if Pmentoge rounds to xero. 

* SigniHcmitiy different front tentroi tity public schools. 

** Significantly different from notion (public schools). 

^ For comportsen, at eighth grade 61 percent of students in central dty pubb'e schools and 36 percent in public schools no^'onolly were non-\^ite. Also, 47 percent of students in central city puidk schools 
and 34 percent in pubHc schools noUonaOy were eligible for hee/reduced'prke school lundi. 

NOTE: Percentages below and at or dtove Basic may not add to 1 00, due to rounding. 

SOURCE: U.S. Deportment of Education, institute of fehicadon Scienms, Nadonal Cent» for feiucodon Stotistics, Notional Assessment of Educational Progress (NAEP), 2002 Triol Urbon DistricI Reading 
Assesmtent. 
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Appendix A 

Identification, Exclusion, and 
Accommodation Rates for 
Special-Needs Students 

The following appendix presents the percentages of students 
with disabilities and limited Emglish proficient students who 
were identified, exxluded, and assessed with accommodations 
in the Trial Urban District Assessment. In interpreting the 
performance results presented in this report, it is important 
to consider the magnitude of rates of identification and 
exclusion of these special-needs students. At grade 4, there 
was considerable variation in exclusion rates for limited 
English proficient students among the districts. 
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Students with Disabilities 
(SD) and/or Limited English 
Proificient (LEP) Students 

It is N^\EP’s intent to assess all selected 
students from the target population. Tliere- 
fore, ev’-ery effort is made to ensure that all 
selected students who ;u‘e capable of partici- 
pating in the assessment are assessed. Some 
students sampled for participation in NAEP 
can be excluded from the sample according 
to c;arefully defined cnteria. These cnteria 
were revised in 1996 to communicate more 
cle:u:ly a presumption of inclusion except 
under special circumstances. According to 
these criteria, students who have an Indi- 
vidualized Education Program (lEP) or are 
protected under section 504 of the Rehabili- 
tation Act of 1973 are to be included in the 
NiVEP assessment except in the following 
cases: 

• the school’s lEP team determines that the 
student cannot participate; 

• the student’s cognitive functioning is so 
severely impaired that she or he cannot 
participate; 

• the student’s TEP requires that the 
student has to be tested with an 
accommodation or adaptation that 
NAEP does not allow and the student 
cannot demonstrate his or her knowledge 
without tliat accommodation. 

All LEP students who received academic 
instruction in English for three years or 
more were to be included in the assessment. 
Those LEP students who received instruc- 
tion in B’nglish for fewer than three years 
were to be included unless school staff 
judged them to be incapable of participat- 
ing in the assessment in English. 



Participation of SD and/or LEP 
Students in the NAEP Samples 

Testing all S 2 impled students is the best way 
for N^'VEP to ensure that the statistics 
generated by the assessment ;ire as represen- 
tative as possible of the performance of the 
entire national population ;ind the popula- 
tions of participating jurisdictions and 
districts. However, all groups of students 
include certain proportions that cannot be 
tested in large-scale assessments (such as 
students who have profound mental disabili- 
ties) or who can only be rested through the 
use of ‘'accommodations, ” such as extra 
time, one-on-one administration, or use of 
magnifying equipment. Some students with 
disabilities and some LEP students cannot 
show on a written test what they know and 
can do unless they are provided with ac- 
commodations. Wlaen such accommoda- 
tions are not allowed, students requiring 
such adjustments are often excluded from 
large-scale assessments such as N^AEP. 
Section 504 of the Rehabilitation Act of 
1973 requires that, when students widi 
disabilities are tested, schools must provide 
them with appropriate accommodations so 
that the test results accurately reflect stu- 
dents’ achievement. The provision of 
accommodations has become more com- 
mon with the passage of the 1997 Individu- 
als with Disabilities Education Act (IDEA), 
which led schools and states to identify 
increasing proportions of students iis needing 
accommodations on assessments in order to 
best show what they know and can do.^ 



^ Office of Speciid Education Programs. (1997). Nineteenth Amtual Report to Con^^ress on the Implementation of the 
Indiviibtals With Disabilities Eiinmtion Ad. Wasliington, DC; U S. Department of Education. 
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In addition, as the proportion of limited 
English proficient students in the popula- 
tion has increased, some states have started 
offering accommodations, such as transla- 
tions of assessments or the use of bilingual 
dictionaries, as part of iossessments. 

Before 1996, NAEP did not allow, any 
testing under nonstandiird conditions (i.e., 
accommodations were not permitted). At 
that time, NAEP samples were able to 
include almost all saiTipled students in 
standard assessment sessions. However, as 
the influence of IDEA grew more wide- 
spread, the failure to provide accommoda- 
tions led to increasing levels of exclusion in 
the assessment. N^AEP’s response was to 
begin a reseiirch program to assess the 
impact on NhAEP’s scales and trends of 
permitting accommodations for those 
students who normally received them in 
classroom assessments. Beginning in 2002, 
N..AEP uses only die more inclusive samples 
in which assessment accommodations are 
permitted. Consequently the data reported 
here for urban districts represent samples in 
which accommodations were permitted. 



Percentages of SD and/or LEP students 
for the Trial Urban District Assessment are 
presented in table A.l. The data in this table 
include the percentages of students identified 
as SD and/or LEP, the percentage of 
students excluded^ and die percentage of 
accommodated SD and/or LEP students. 
Different rates of exclusion may influence 
the meaning of district comparisons. Thus, 
exclusion data should be carefully reviewed 
in this context. 

The percentage of fourth-graders in 
participating urban districts excluded from 
die assessment ranged from two percent in 
Atlanta to 17 percent in Houston. At grade 
4, the percentage ot students excluded in 
the national public school sample was seven 
percent, and the percentage of students 
excluded in the national central city public 
school sample was eight percent. The 
percentage of eighth-graders in participat- 
ing urban districts excluded from the 
assessment ranged from tw^o percent in 
Atlanta to nine percent in New York City. 

At grade 8, the percentage of students 
excluded in the national public school 
sample was six percent, and the percentage 
of students excluded in the national central 
city public school sample was seven percent. 




NAEP 2002 READING TRIAL URBAN DISTRICT ASSESSMENT 

52 



APPENDIX A • 



39 



Toble A J Percentage of students with disabilities and limited English proficient students identified, excluded, ond assessed with 
accommodations, grades 4 ond 8 public schools; By urban district, 2002 





Students with disobilifies ond/or 
limited English pro^dent students 


Students with disabilities 


1 

limited English profldent students 




Identified 


Excluded 


Assessed 

with 

accommodations 


Identified 


Excluded 


Assessed 

with 

accommodations 


Identified 


Exduded 


Assessed 

with 

accommodations 


1 Grade 4 




















Notion (Public) 


21 


7 


4 


13 


5 


4 


9 


2 


1 


Central city (Public) ^ 


25 


8 


4 


13 


6 


3 


14 


4 


1 


Atlanta 


8 


2 


1 


5 


1 


1 


4 


1 


0 


Chicago 


30 


9 


5 


16 


4 


4 


19 


7 


2 


District of Columbia 


19 


8 


5 


14 


7 


4 


7 


3 


2 


Houston 


43 


17 


1 


12 


4 


1 


36 


16 


# 


Los Angeles 


51 


8 


2 


11 


3 


2 


46 


6 


1 


New York City ^ 


22 


8 


8 


14 


5 


6 


11 


6 


3 


1 Grade S 




















Notion (Public) 


18 


6 


4 


13 


5 


4 


1 6 


2 


1 


Central city (Public) ^ 


22 


7 


4 


14 


5 


3 


10 


3 


1 


Atlanta 


6 


2 


1 


5 


1 


1 


1 


# 


# 


Chicago 


21 


6 


7 


15 


3 


6 


8 


4 


1 


District of Columbia 


21 


7 


8 


16 


6 


7 


5 


2 


2 


Houston 


27 


7 


0 


15 


5 


0 


16 


4 


0 


Los Angeles 


35 


5 


2 


12 


3 


2 


30 


5 


1 



m not met. 

in public schooh nofiomilly were non-Vi^iie. Also, 6) penent ol students m central city public scbook and 43 percent In public schools 

notionolly were eligible for frep/reduced-price school lunch. 

^ For comporison, ot eighth grode 61 percent of students in central city public schools ond 36 percent in public schools notionolly were non-White. Aka, 47 percent of students in central dty publk schools and 34 percent in public schools 
notionolly were eligible for kea/reduced-price school lunch. 

NOTF: Some students were identified os both SD ond LEP. Such students would be included in both the SD ond liP portions of the table. 

SOURCE: U.S. Deportment of Education, Institute of Educotion Sciences, Notionoi Cotter for Education Statistics, Notional Assessment of Educotionol Progress (NAEP), 2002 Reading Triot Urban Oistrkt Assessment. 



# Percentoge rounds to zero. 

^ Although deemed sufficient for reporting, the target response rote specified in the NAEP guidelines i 
^ For comparison, at fourth grade 65 percent of students in centred dty public schook and 40 percem 
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In the 2002 national combined sample 
(public and nonpublic schools), 6 percent 
of students at grade 4, and 5 percent of 
students at grade 8 were excluded from the 
assessment (see table A.IO of the The 
Nation's Report Card: Reading 2002) r Across 
the various states and jurisdictions that 
participated in the 2002 state assessment, 
the percentage of students excluded ranged 
from 3 to 12 percent at grade 4 (see table 
A.11 in the national report card for reading 
2002) <md from 2 to 10 percent at grade 8 
(see table A.12 in the national report card). 



Types of Accommodations 
Permitted 

Table A.2 displays the percentages of 
SD and/ or LEP students assessed with the 
variety of available accommodations at 
grade 4. Table A.3 shows the comparable 
data for grade 8. It should be noted that 
students assessed with accommodations 
typically received some combination of 
accommodations. The numbers iind per- 
centages presented in the table reflect only 
the prim',ary accommodation provided. For 
example, students assessed in small groups 



^ Grigg, W S., Daaue, M C, Jin, Y, and Campbell,}. R. (2003). The Natio/rs Report Card: Reading 2002 
(NCES 2003—521). Washington, DC: US. Department of Education, Institute of Education Sciences, 
Naliomd Center for Education Statistics. 
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(as compared wirii standard N^\EP sessions 
of about 30 students) usually received 
extended time. In one-on-one administra- 
tions. students often received assistmce in 
recording answers and were afforded extra 
time. Extended time was considered the 
primary accommodation only when it was 
the sole accommodation provided. 

The assessment did not allow some 
accommodations that were permitted in 
certain states in past assessments. Some 



states have allowed questions and, in some 
cases, reading passages to be read aloud to 
the students. In designing the reading 
assessment, reading .aloud as an accommo- 
dation Wiis viewed ^is changing the nature of 
the construct being me;isured, and hence 
was not permitted. Because NAEP consid- 
ers the domiiin of its reading assessment to 
be reading in Englisli, no attempt was made 
to provide an alternate language version of 
the assessment, md die use of bilimmal 
dictionaries was not permitted. 



Table A.2 Students with disabilities and limited English proficient students assessed with accommodations, 
by type of primary accommodation, grade 4; By urban district, 2002 





Lorge-print 

book 


Extended 

time 


Weighted percentage of students sampled 

Small Scribe/ 

group One-on-ofie computer 


1 

Other 


$0- dttd/or LEP" students 


Notion (Public) 


0.04 


1.57 


2.23 


0.08 


0.06 


0.03 


Central city (Public) ^ 


0.02 


2.09 


1.96 


0.05 


0.04 


0.03 


Atlanta 


0.06 


0.77 


0.06 


# 


# 


# 


Chicago 


# 


3.67 


1.30 


# 


0.05 


0.10 


District of Columbia 


# 


2.80 


2.45 


# 


# 


0.07 


Houston 


# 


0.32 


0.73 


# 


# 


# 


Los Angeles 


0.05 


1.08 


1.23 


0.12 


# 


# 


New York Gty ^ 


# 


6.61 


1.49 


# 


# 


# 


SD’ students 


Nation (Public) 


0.04 


1.24 


2.08 


0.08 


0.06 


0.03 


Central city (Public) ^ 


0.02 


1.53 


1.83 


0.04 


0.04 


0.02 


Atlanta 


0.06 


0.77 


0.06 


# 


# 


# 


Chicago 


# 


2.51 


1.30 


# 


0.05 


O.IO 


District of Columbia 


# 


2.11 


1.63 




# 


0.07 


Houston 


# 


0.32 


0.73 


# 


# 


# 


Los Angeles 


0.05 


0.98 


1.05 


0.12 


# 


# 


New York City ^ 


# 


5.18 


0.92 


# 


# 


# 


LEP- students 


Nation (Public) 




0.44 


0.25 


0.01 


# 


0.01 


Central dty (Public) ^ 


# 


0.79 


0.25 


0.01 


# 


0.01 


Atlanta 


# 


# 


# 


# 


# 


# 


Chicago 


# 


1.88 


0.35 


# 


# 


0.05 


District of Columbia 


# 


0.77 


1.03 


# 


# 


0.03 


Houston 


# 


0.16 


# 


# 


# 


# 


Los Angeles 


# 


0.56 


0.82 


0.07 


# 


# 


New York Gty ^ 


# 


1.94 


0.66 


# 


# 


# 



# Percentoge rounds to zero. 

^ Alltiough deemed sufficient for reporting, the torget response rote specified in the NAEP guidelines vras not met. 

^ Students with dhoi^ties. 

^ Limited biglish proficient students. 

^ For comporison, ot fiturth grade 65 p^nt of students in central city public schools and 40 percent In pubhc schools notjonoOy were non-While. Also, 61 percent of students in centred city public 
schools and 43 percent in public schools notlonolly were eligiUe for free/reduced-price school lunch. 

NOTE: Some students were identified os both SO ond LB*. Such students would be included in both the SO and LB’ portions of the tohle. 

SOURCE: Ui. Department of EducoNon, Institute of Educo^on Sciences, No^onol Center for Educo^on Statistics, Notiond Assessment of Educotiimd Progress (NAEP), 2002 Trid Urban District Reodng 
Assessment. 
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Table A.3 Students with disabtllties and limited English proficient students assessed with accommodations, by type of 
primary accommodotion, grode 8: By urban district, 2002 





Lorge-prlnt 

book 


Extended 

time 


Weighted percentage of students sampled 

Smell Scribe/ 

group One-on-one computer 


— 1 

Other 


$0- and/or lEF^ students 


Nation (Public) 


0.02 


2.06 


1.69 


0.05 


0.03 


0.04 


Centrol dty (Public) ^ 


0.01 


2.25 


1.52 


0.05 


0.01 


0.06 


Atlonta 


# 


# 


1.00 


# 


# 


# 


Chicago 


# 


2.50 


4.05 


# 


# 


# 


District of Columbia 


0.06 


6.02 


1.95 


# 


# 


# 


Houston 


# 


# 


# 


# 


# 


# 


Los Angeles 


# 


1.05 


1.18 


0.19 


# 


# 


SD’ students 


Notion (Public) 


0.02 


1.82 


1.61 


0.05 


0.03 


0.04 


Central dty (Public) ^ 


0.01 


1.80 


1.45 


0.05 


0.01 


0.06 


Atlanto 


# 


# 


1.00 


# 


# 


# 


Chicago 


# 


2.19 


3.95 


# 


# 


# 


District of Columbia 


0.06 


4.58 


1.95 


# 


# 


# 


Houston 


# 


# 


# 


# 


# 


# 


Los Angeles 


# 


1.05 


1.18 


0.19 


# 


# 


lEP^ students 


Notion (Public) 


# 


0.39 


0.14 


# 


# 


# 


Centrol dty (Public) ^ 


# 


0.72 


0.21 


# 


# 


# 


Atlanto 


# 


# 


# 


# 


# 


# 


Chicago 


# 


0.65 


0.38 


# 


# 


# 


District of Columblo 


# 


1.86 


0.06 


# 


# 


# 


Houston 


# 


# 


# 


# 


# 


# 


Los Angeles 


# 


0.58 


0.88 


# 


# 


# 



# Pertentogo rounds to zero. 

^ Students with d^biiities. 

^ Limited bigiish proHdent students. 

^ For (omporison, oi eighth grode 61 percent of students in cenlro! dty public schods (md 36 percent in public schools notionoily were non-White. Also, M percent of students in central city public schools 
ond 34 percent in public schools nohonoliy were eligiUe fm free/reducsd-price school lunch. 

NOTE: Some students were identified os both SD ond IH Such students would be included in both the SO ond LEP portions of the table. 

SOURCE: U5. Deportment of Blucotion, Institute of Blucotion Sciences, Notionol Center for Blucotion Smtistics, Notional Assessment of Educotionol Progress (NAEP), 2002 Trio! Urbon District Reoding 
Assessment. 
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Investigating the Potential Effects 
of Exclusion Rates on Assessment 
Results 

In considering the effects of exclusion 
rates on assessment results, at least two 
major issues are evident. First, if exclusion 
rates vary substantially across assessment 
years, then the ability to report trends (i.e., 
compare results between years) may be 
affected by the fict that the results from 
different years are based on different pro- 
portions of the population. Second, the 
variation in exclusion rates among states, 
jurisdictions, and school districts may 



threaten the comparison of results within a 
given year, because the results for different 
districts, states, or jurisdictions are based on 
different proportions of the populations. 

NCES has funded research investigating 
ways in which excluded students might be 
included in the estimation of scores for 
total populations. NCES has also commis- 
sioned studies on the impact of assessment 
accommodations on overall scores. A 
detailed discussion of some of tliese estima- 
tion procedures is included in appendix A 
of the full report card.^ 



^ Giigg, W S., Daaiie, M. Y., aad Campbell, J. R. (2003). T/jtf Na/ion^s Report Card: Reading 2002 

(NCES 2003—521). Wasliiugtou, DC: US. Departineut of Education, Institute of Education Sciences, 
National Center for Education Statistics. 
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Appendix B 

District-Level Contextual Variables 



To help place results from the NAEP 2002 Trial Urban 
District Assessment into context, this appendix presents 
selected district-level data from sources other than NAEP. 

These data are taken from the Public Elementary/ Secondary 
School Universe Survey, 2000— 01 , and Eocal Education Agency 
Universe Survey, 2000— 01 . The interested reader can access 
most of the data presented in this appendix in the NCES 
report, Characteristics of the 100 Eargest Public Elementary 
and Secondary School Districts in the United States at 
http:/ / nces.ed.gov/ pubs2001 /100_largest/ tab_fig.asp. 
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Tobie B.l Number of students, high-school completers, teachers, and schools, from non-NAEP sources: 
By urban district, school year 2000-01 







Number of 


Number of 


1 




Number of 


1999-2000 


full-time equivalent 


Number of 




students' 


completers^ 


teachers 


schools 


NotioriQl 


47,086,931 


2,548,076 


2,841,677 


93,344 


Central dty 


13,523,126 


610,467 


808,288 


22,310 


Atlontc 


58,230 


2,056 


3,950 


98 


Chicago 


435,261 


14,875 


23,935 


602 


Distrkf of Columbia 


68,925 


2,916 


5,044 


165 


Houston 


208,462 


7,735 


11,197 


289 


Los Angeles 


721,346 


27,439 


35,150 


659 


New York City 


1,066,516 


40,827 


65,242 


1,213 



^ Count of students receiving educotionol services from school district may differ somewhot ^om the counts in table 6.4, which reflect the count of smdents from the schools oggregoted up to the schod 
district. 

^ Includes high school diplomo recipients os well os other high school completers (e.g., certihcotes of ottendonce), but does not include high school equivalencies (ex. GEDs). 

SOllRCL' U.S. Deportment of Educotion, Notional Center for &iucation Statistics, Common Core of Dato, locol &iucohon Agency Universe Surray, 2000-01, vmion lo. 



Table B.2 Poverty rate and federal funding, from non-NAEP sources: By urban district, 
1996-97, 1997-98,2001-02 





5- to 1 7-yenr olds, 
1999-2000 




Revenue, 

1999-2000 

(In thousands) 




Title 1 allocations, I 
2001-02 

(in thousands) 












Federal as 


Federal 










Percent 






percent 


revenue 


Basic 


Concentration 




Population 


in poverty 


Total 


Federal 


of total 


per student 


grants 


grants 


Atlonto 


66,131 


34.6 


$695,919 


$56,558 


8.1 


$95.17 


$752 


$20,824 


Chkogo 


540,667 


24.8 


3,604,873 


539,567 


15 


124.97 


963 


137,865 


District of Columbia 


82,456 


27.8 


881,423 


184,825 


21 


261.19 


1,509 


21,341 


Houston 


230,514 


26 


1,469,074 


152,679 


10.4 


72.8 


585 


50,084 


Los Angeles 


863,656 


28.3 


5,757,819 


592,508 


10.3 


83.45 


668 


178,088 


New York City 


1,397,739 


28.3 


10,945,650 


1,119,944 


10.2 


104.11 


812 


397,541 



NOTE: Federal revenue per student based on foil enrollment collected by the Bureau of the ^nsus. Detoil moy not sum to totoH, becouse of to n 
SOURCE: U.S. Department of Educotion, Notionol Center fm Education Stotistics, Common Cm'e of Doto survey; ond U.S. Deportment of Commm^ce, ‘Survey of Local Government Rnonces.* 
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Table B.3 Number of public elementary and secondory schools, by type of school from non-NAEP sources: 
By urban district, school year 2000H)1 



Type of school ^ 





Total number 
of schools 


Regular 


Special 

education 


Vocational 

education 


Alternative 


National 


93,344 


85,493 


2,008 


1,025 


4,818 


Central city 


22,310 


20,141 


536 


173 


1,460 


Atlanta 


98 


95 


0 


0 


3 


Chicago 


602 


578 


24 


0 


0 


Distrkt of Columbia 


165 


150 


10 


0 


5 


Houston 


289 


264 


1 


3 


21 


Los Angeles 


659 


582 


18 


0 


59 


New York City 


1,213 


1,106 


8 


18 


81 



^ Type of school is o mutuolly exclusive cotegory on the Common Core of Doto. There ore coses in which spedol educotion, vocotionol education, ond oitmotive progroms redde in other types of schools. 
NOTE: Types of schools ore defined m the following woy on the Common Core of Doto: Regulor sdtool— A pubhc eiementory/secondory school that does not focus primardy on vocational, special, or 
oiternotive educotion. NAEP Is conducted only in regulor schools. Special educotion school— A public elementory/secondary school that (o) focuses primorily on specid education, including instruchon for 
any of the following: hord of hearing, deaf, speech-impaired, health-impoired, onhapethcally impotred, mentally recorded, seriously emotionoHy disturbed, mulH4iondicapp8d, visuoSy handicapped, 
deaf and blind; and (b) odopts curriculum, materiob, or instruction for students served. Vocadanal education school— A public eiementary/seccmdory school that focuses primarily on vocotionol 
educotion, ond provides education ond Iroining in one or more semi-slulted or technical operotions. Altemotive education school— A public eiementary/secondory school that (o) addresses the needs of 
students thot typicoily connot be met in o regulor school; (b) providos nontraditionol education; (c) serves os on adjunct to o regular school; and (d) falls outside of the categories of regular, special 
education, or vocotionol educotion. 

SOURCE: Ui. Deportment of Educohon, Nohonol Center for Educotion Statistics, Comnmn Core of Dom, Public Semen tory/Secondory School Univose Survey, 2000-01, version la. 
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Table B.4 Number of students in public elementory and secondory schools, by type of school, from non-NAEP sources: 
By urbon district, school yeor 2000-01 





Total number 
of students^ 


Regular 


Notional 


47,094,888 


46,228,904 


Central dty 


13,522,154 


13,152,151 


Atlonta 


58,230 


56,896 


Chicago 


435,261 


431,553 


Distncf of Coiumbio 


68,925 


65,285 


Houston 


208,462 


204,042 


los Angeles 


721,346 


704,932 


New York City 


1,066,945 


1,009,319 



Type of school^ 


Special 

education 


Vocotionol 

education 


Other ond 
oitemative 


174,577 


199,669 


491,738 


73,387 


75,953 


220,663 


t 


t 


1,334 


3,708 


t 


t 


2,772 


t 


868 


19 


906 


3,495 


4,480 


t 


11,934 


1,688 


25,409 


30,529 



f No students reported in membership for this type of sthod. 

^ Typo of school is a mutuoUy exclusive cotegory on the Common Core of Doto. Ihere ore coses in which ^eciol educotion, vocational education, and alternative programs resido kt other types of schoob. 
^ Student distribution by type of school is hosed on membership in Ihe schools of the school dhtrict. Quints moy vory from those in mbie B.l . 

NOTH: Types of schools ore deBned in ihe following woy on the Common Core of Data: Regular schod—A pul^c elemenlory/secondory schcml that does not focus primarily on vocotlonal. special, or 
olternotive educotion. NAHP is conihicted only in regulor schools, fecial educotion school— A public eiementory/secondary school ihot (o) focuses primorily on special educotion. including instruction for 
any of the foBowing: hord of hearing, deof, speech'impoired. heolth'lmpoired. orthopedicolly inguifred. rnenmlly retorded, seriously emotionolly disturbed. nrulH-hondicopped. visuolly hondicof^ed. 
deaf ond blind; ond (b) odopts curriculum, moteriols, or instruction for students served. Vocoticmol educotion school— A puUic eiementory/secondary school thot focuses primarily on vocotionol 
education, ond provides educotion and training in one or more seim-skilled or technicol operations. Alternative education school— A public elementory/secandory school that (o) oddresses the needs of 
students thot typicoBy cannot be met in o regulor sthod; (b) prondes nontroditionol education; (c) serves os on adjunct to a regular school; and (d) folk outside of the cotegories of regular, special 
educotion, or vocotionol education. 

SOURCE: U.S. Deportment of Education, Notionol Center for Education Statistic, Common Qire of Doto, Public Efementory/Seconchiry School Universe Survey, 2000-0), version io, and Locd Education 
Agency Universe Survey. 2000-01, version lo. 
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Table i.S Percentage of minority students, and number of public elementary and secondary schools within specified 
ranges of minority student percentages, from non'’NAEP sources: By urban district, school year 2000-01 





Percentage 
of minority 
students 
enrolled 


Number of 
schools with 
students ^ 


Number of schools with 

0~20 21-40 

percent percent 


a minority student’ percentage of 

41-60 61-80 81-100 

percent percent percent 


Notbnai 


39.0 


89,110 


43,827 


14,023 


9,866 


7,142 


14,252 


Central city 


63.0 


21,513 


3,534 


3,165 


3,370 


3,088 


8,356 


Atlonto 


93.2 


98 


2 


2 


2 


5 


87 


Chicago 


90.4 


596 


1 


20 


26 


51 


498 


IHstrid of Columlna 


95.5 


165 


0 


5 


3 


5 


152 


Houston 


90.0 


289 


1 


4 


17 


20 


247 


Los Angeles 


90.1 


659 


4 


16 


50 


88 


501 


Hew York City 


84.7 


1,207 


29 


39 


114 


139 


886 



^ Minority students, in this table, indudes ail race/ethnidty categories except White, non-Kisponic. 

^ tndudes only schools for which student memberdiip by roce/ethnicity wcb reported. 

SOURCE: U.S. Department of Education, National Center far Mutation Statistics, Common Core of Data, Public Elementary/Secandary School Universe Survey, 2000-4)1, version la, 
and Local Education Agency Universe Survey, 2000-01, version la. 



Table B.6 Number of public elementary and secondary schools, by instructional level, from non-NAEP sources: 
By urbon district, school yeor 2000-01 





Number of schools 




Number of schools by instructional level’ 






with students^ 


Primary 


Middle 


High 


Other 


National 


90,711 


52,285 


15,584 


17,280 


5,562 


Centro! city 


21,900 


13,962 


3,318 


3,081 


1,539 


Atlonta 


98 


70 


15 


12 


1 


Chicago 


596 


472 


21 


72 


31 


Distiict of Columlna 


165 


112 


10 


16 


27 


Houston 


289 


197 


43 


35 


14 


Los Angeles 


659 


456 


75 


105 


23 


New York City 


1,207 


692 


210 


184 


121 



^ Instructional level is based on the lowest and highest grade in a school: Primory schools t»gin between prekindergarten ond ^ade 3 cmd may go os high as grade 8. Middle sdiook have grade spans 
ranging from as low as grode A to as high as grade 9. High schocds start at grade 7 or higher ond must extend through grade 12. Otl^r schools indcKle all other ^de combinalions, including 
mekindergorten, kindergorten, or 1-12, and ungraded sdtools. 

' Indudes only schools for which student membership reported. 

SOURCE: U.S. Department af Education, National Center for ^ucotlon Statistics, Common Core of Data, Pubhc Elementory/Serandary School Universe ^rvey, 2000-01, version 1o, 
and Locoi Education Agency Unhmrse Survey, 2000-01, version la. 
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Tabie BJ Median pupil/teacher ratios in public elementary and secondary schools, by instructional level, from non* 
NAEP sources: By urban district, school year 2000*'-'01 



Median pupil/teacher ratio’ 

By school instructional leveP 





Overall 


Primary 


Middle 


High 


Other 


National 


15.9 


16.3 


15.7 


15.1 


12.1 


Centro! dty 


16.6 


16.7 


16.0 


17.1 


11.2 


Atlnnta 


14.2 


13.7 


16.2 


16.2 


13.2 


Chicago 


18.4 


18.9 


17.3 


15.5 


11.9 


Bistrkt of ColumlHa 


13.5 


13.7 


13.6 


13.6 


8.7 


Houston 


18.5 


18.7 


18.7 


18.6 


3.3 


Los Angeles 


19.6 


19.2 


23.6 


22.7 


10.4 


New York City 


16.1 


15.7 


16.7 


18.7 


6.3 



’ Includes only those schools whose student membership wos greoter than z^o. 

^ Instructionol level is bosed on the lowest and highest grade offered in n school. Primary schools begin between prekindergarten and grade 3 ond may go as high as grade 8. Middle schools have grode 
spans ronging from os low os grade 4 to os high os grade 9. Hqh schools stort at grade 7 or higlsr and must extend through grade 12. Other schools include alt other grade combinotions, including 
prekindergarten, kindergarten, or t-I2, ond ungraded schools. 

SOURCE: Ij.S. Oeportmenl of Educotion, Naticmal Cent^ for Education Stotishcs, Common Core of Data, Public Elemenlcn^/Secondory School Universe Survey, 20{]{M)1, version 1o, 
ond Loco] Educotion Agency Universe Survey, 200(M)1, version lo. 



BEST COPY AVAILABLE 




APPENDIX B • NAEP 2002 READING TRIAL URBAN DISTRICT ASSESSMENT 



Table E.S Percentage of staff In public elementary and secondary schools, by type of staff, from non-NAEP sources: 
By urban district, school year 2000H)1 



Percent of full-time equivalent staff 





Total 




Instructional 


Guidance 


Library 


District 


School 


Other 




staff 


Teachers 


support 


counselors 


media staff 


administrators 


administrators 


staff 


Natlonol 


5,397,788 


52.6 


11.4 


1.8 


1.5 


1.0 


2.5 


29.1 


Central city 


1,513,730 


53.4 


11.2 


1.7 


1.3 


0.8 


2.5 


29.1 


Atlonto 


7,552 


52.3 


12.7 


2.0 


1.3 


0.5 


2.5 


28.8 


Chicago 


28,687 ^ 


83.4 


2.9 


2.9 


1.7 


1.7 


3.1 


4.4 


f^stnct of Columbia 


10,808 


46.7 


10.8 


1.9 


1.3 


0.1 


2.5 


36.8 


Houston 


24,820 


45.1 


10.2 


1.2 


0.9 


0.1 


2.3 


40.2 


Los Angeles 


66,598 


52.8 


14.5 


1.4 


0.1 


0.6 


1.9 


28.7 


New York City 


100,198 


65.1 


0.8 


1.9 


0.7 


0.7 


2.9 


27.9 



^ Ihe non'teoching staff (oteg(me$ may ba underrepresented. 

NOTE: Percmitaga may not add to 100, due to rounding. 

SOURCE: Ui. Department of Education, Notional Center for Education Statistics, Common Core of Data, Local Education Agency Universe Survey, 2000-01, version 1 o. 
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Table 8.9 Percentage of reported students eligible for free/reduced^prlce school lunch and percentage of students in 
each racial/ethnic category, from non-NAEP sources: By urban district, school year 2000-01 



Students eligible for Raclal/ethnic composition of district as 

free/reduced-price lunch ' a percentage of students enrolled 





Number of 
schools with 
students 


Percentage 
of students 


Percentage 
of schools 
reporting 


White 


Block 


Hispanic 


Asion/ 

Pocific 

Islonder 


American 

Indian/ 

Alaska 

Native 


Nafiotid 


90,711 


38.6 


83.8 


61.0 


17.0 


16.6 


4.2 


1.2 


Central city 


21,900 


53.6 


82.9 


36.8 


29.2 


27.7 


5.5 


0.9 


Atlanto 


98 


76.4 


100.0 


6.8 


89.5 


2.8 


0.9 


0.1 


Chicago 


596 


- 


0.0 


9.6 


52.0 


34.9 


3.3 


0.2 


DIsfrIc! of Columbia 


165 


70.0 


98.8 


4.5 


84.6 


9.2 


1.6 


0.1 


Houston 


289 


70.7 


99.7 


10.0 


32.1 


55.0 


2.9 


0.1 


Los Angelos 


659 


73.5 


99.8 


9.9 


12.8 


70.8 


6.3 


0.3 


New York Gty 


1,207 


71.9 


100.0 


15.3 


34.9 


37.8 


11.7 


0.3 



— Not ovoilobls. 

^ Hiese pfircentoges should be interpreted with coution; jurisdictions may not hove reported students eligible for reduced-price meals, ond a number of jurisdictions reported participation instead of 
eligibility data, which may n at be strictly camparable. Percentoges are based an those schools that reported. 

^ Includes only schools for which student membership wos reported. 

SOURCE: U.S. Department af Educahon, Notional Center for tiucation Stotishcs, Common Care of Dom, Public Elementary/Secondary School Universe Survey, 2000-0], version to, 
and Local Biucotion Agency Universe Survey, 2000-01, version la. 
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C Appendix C 

Overview of Procedures Used 

for the NAEP 2002 Trial Urban District 

Reading Assessment 

This appendix provides an overview of die N AHP 2002 
reading assessment’s primary components — framework, 
development, administration, scoring, and analysis. A more 
extensive review of the procedures and methods used in the 
national and state reading assessments can be found in the 
assessment procedures sections of the NAEP web site 
(http:// nces.ed.gov/nationsreportcard). 

The NAEP 2002 Reading Assessment 

The National Assessment Governing Board (NAGB), 
created by Congress in 1988, is responsible for formulating 
policy for NAEP. NAGB is specifically charged with 
developing assessment objectives and test specifications. The 
design of the NAEiP 2002 reading assessment follows the 
guidelines first provided in the framework developed for tlie 
1992 assessment.^ The framework underlying the 1992, 1994, 
1998, 2000 (fourth grade only), and 2002 reading assessments 
reflects the expert opinions of educators and researchers 
about reading. Its purpose is to present an overview of tlie 
most essential outcomes of students’ reading education. lEe 
development of this framework and the specifications that 
guided the development of the assessment involved the 
critical input of hundreds of individuals across the country, 
including representatives of national education organizations, 
teachers, parents, policymakers, business leaders, and the 
interested general public. Hie framework development 



^ Nalioual Assess uieiit Govemiug Board. (2002). Ksacling Framework for the 2003 National 
Assessment of Educational Progress. Washington, DC: Aiithoi. 
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process was managed by the Council of 
Chief State School Officers (CCSSO) for 
NAGB. 

The framework sets forth a broad defini- 
tion of ''reading literacy’’ — de%’^eloping a 
general understanding of a text, thinking 
about a text in different ways, and using a 
variety of text types for different purposes. 
In addition, the framework views reading as 
an interactive and constructive process 
involving the reader, the text, and the 
context of the reading experience. For 
example, readers may read stories to enjoy 
and appreciate the human experience, study 
science texts to form new hypotheses about 
knowledge, or use maps to gain information 
about specific places. Ni\ILP reflects current 
definitions of literacy by ditferentiating 
among three contexts for reading and four 
aspects of reading. Contexts for reading :ind 
aspects of reading make up the foundation 
of the NAEP reading assessment. 

The “contexts for reading” dimension of 
tlie NxAEP reading framework provides 
guidance for the types of texts to be in- 
cluded in the assessment. i\J though many 
commonalities exist among the different 
reading texts, they do lead to real differ- 
ences in what readers do. For example, 
when readingfor literary experience, readers 
make complex, abstract summaries and 
identify major themes. They describe the 
interactions of various literary elements 
(e.g., setting, plot, characters, and theme). 
When readingfor information, readers critically 
judge the form and content of the text and 
explain their judgments. They also look for 
specific pieces of information. When reading 



to peform a task, readers search quickly for 
specific pieces of information. 

The “aspects of reading” dimension of 
the N^AEP reading framework provides 
guidance for the ty^pes of comprehension 
questions to be included in the assessment. 
The four aspects are forming a genera! 
understandings 2) dei 'doping interpretation, 3) 
making reader! connections, and 4) examining 
content and stmctur'e. These four aspects 
represent different ways in which readers 
develop understanding of a text. In foming a 
general understanding, readers must consider 
die text as a whole and provide a global 
understanding of it. As readers engage in 
developing interpretation, they must extend 
initial impressions in order to develop a 
more complete understanding ot what was 
read. This involves linking information 
across parts of a text or focusing on specific 
information. VCTien making reader! text 
connections, the reader must connect informa- 
tion in the text with knowledge and experi- 
ence. This might include applying ideas in 
the text to die real world. Finally, examining 
content and structure critically evaluat- 

ing, comparing and contrasting, and under- 
standing the effect of different text features 
and autiiorial devices. 

Figure C.l demonstrates the relationship 
between these reading contexts and aspects 
of reading in die reading assessment. 

Included in the figure are sample questions 
that illustrate how each aspect of reading is 
assessed within each reading context. (Note 
that reading to perform a task is not assessed at 
grade 4.) 
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Figure CJ Sample NAEP questions, by aspects of reading and contexts for reading specified in the reading fromework 



Aspect of Reoding 



Context for Reading 


Forming a 

general understanding 


Developing 

interpretation 


Making 

reader/text connections 


Examining 

content and structure 


Reading for literary 
experience 


what is the 
story/plot about? 


How did this character 
change from the 
beginning to the end of 
the story? 


What other character 
that you have read 
about had a similar 
problem? 


What is the mood of this 
story and how does the 
author use language to 
achieve it? 


Reading for information 


What point is the author 
making about this topic? 


What caused this change? 


What other event in 
history or recent news is 
similar to this ane? 


Is this author biased? 
Support your answer 
with information about 
this article. 


Reading to perform a task 


What time can you get a 
nonstop flight to X? 


What must you do before 
step 3? 


Describe a situation in 
which you would omit 
step 5. 


Is the information in this 
brochure easy to use? 



SOURCE: National Assessment Governing Board. (2002). Reoding fromework for the W03 Hatianol Assesment of Sdutationai Progress. Washington, DC: Author. 



The assessment framework specifies not 
only the particular dimensions of reading 
literacy to be measured, but also the per- 
centage of assessment cpestions that should 
be devoted to each. The tirget percentage 
distribution for contexts of reading iind 
aspects of reading as specified in the frame- 
work, along with the actuiU percentage 
distribution in the assessment, are presented 
in tables C.l and C.2. 



The actual content of the assessment 
varied from the targeted distribution, with 
reading for literary experience falling below 
the target proportions and reading for 
information tailing above the tu'get propor- 
tions specified in the fnimework. The 
reading instrument development p«inel 
overseeing the development of the assess- 
ment recognised this variance but felt 
strongly that assessment questions must be 
sensitive to the unique elements of the 
authentic reading materials being used. 
Thus, the distribution of question classifica- 
tions will vary across reading passages and 
reading purposes. 
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Table Cl Target and actual percentage distribution of questions, by context for reading, grades 4 and 8; 2002 . 







Context for Reading 


1 




Reading for 


Reading for 


Reading to 




literary experience 


information 


perform a task 










Target 


55 


45 


t 


Actual 


50 


50 


t 










Target 


40 


40 


20 


Actual 


27 


43 


30 



^ Reoding to porfonn a task was not assessed ot grade 4. 

SOURCE: Ui. Departmem of Edocotion, institute of Education Sciences, National Center for Education Statistics, Notiono! Assessment of Eduoitionol Progress (NAEP), 2002 Reoding Assessment. 



Table C*2 Target ond actual percentage distribution of questions, by aspect of reading, grades 4 and 8: 2002 







Aspect of Reading 


1 




Forming o general 


Making 


Examining 




understanding/ 


reader/text 


content and 




Developing interpretation 


connections 


structure 










Target 


60 


15 


25 


Actual 


59 


18 


24 










Target 


55 


15 


30 


Actual 


54 


18 


28 



NOTE: Actual percentoges ore bosed on the classifications ogreed upon by N/^s instrument Development Panel. It b recognized thot moking dbcrete classifications for these categories b difficult ond 
that independent efforb to classify NAEP questions have led to different results. 

Percentoges may net odd to 100, due to rounding. 

SOURCE: U.S. Department of Edumtion, institute of Education Sciences, Notiond Center for Educotion Stotbtics, Nationoi Assessment of Educotiond Progress (NAEP), 2002 Reading Assessment. 
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The Assessment Design 

Each student who participated in the 
reading assessment received a booklet 
containing three or four sections: a set of 
general background questions, a set of 
subject-specific background questions, and 
one or two sets of questions assessing 
students’ comprehension of a text or texts. 
The sets of questions assessing students’ 
comprehension are referred to as ‘^blocks.” 
Each block contains one or more reading 
passviges and a set of comprehension 
questions. At grade 8, students were given 
either Kvo 25-minute blocks or one 50- 
minute block. At gracie 4, however, only 25- 
minute blocks were used. 

The blocks contain a combination of 
multiple-choice and constructed-response 
questions. Multiple-choice questions require 
students to select the best answer from a set 
of four options. Constructed-response 
questions require students to provide their 
own written response to an open-ended 
question. Short constructed-response 
questions may require a response of only a 
sentence or two for the answer to be con- 
sidered complete. Extended constructed- 
response questions, however, may require a 
response of a paragraph or more for the 
answer to receive full credit. Each con- 
structed-response question has its own 
unique scoring guide that is used by trained 
scorers to rate students’ responses. (See the 
“Data Collection anci Scoring” section of 
this appendix.) 
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The grade 4 assessment consisted of 
eight 25-minute blocks: four blocks of 
“literary” texts and questions vmd four 
blocks of “informative” texts <ind questions. 
Each block contained at least one passage 
corresponding to one of the contexts for 
reading and 9—12 multiple-choice and 
constructed-response questions. In each 
block, one of the constructed-response 
questions required an extended response. As 
a whole, the 2002 fourth -grade assessment 
consisted of 49 multiple-choice questions, 

45 short constructed-response questions, 
and 8 extended constructed-response 
questions. 

The grade 8 assessment consisted of nine 
25-minute blocks (three literary, three 
intormative, and three task-related) and one 
50-minute block (informative). Each block 
contained at least one passage correspond- 
ing to one of the contexts for reading and 8 
to 13 multiple-choice and constructed- 
response questions. Each block contained at 
least one extended constructed-response 
question. As a whole, the eighth-grade 
assessment consisted of 58 multiple-choice 
questions, 68 short constructed-response 
questions, and 15 extended constructed- 
response questions. 

The assessment design allowed maximum 
coverage of reading abilities at each grade, 
while minimizing the time burden for any 
one student. This was accomplished 
through the use of matrix sampling of 
items in which representative samples of 
students took various portions of the entire 
pool of assessment questions. Individual 
students are required to take only a small 
portion, but the 'aggregate results across the 
entire assessment allow for broad reporting 
of re'ading abilities for the targeted 
population. 
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In addition to matrix sampling, the 
assessment design used a procedure for 
distributing blocks across booklets that 
controlled for position and context effects. 
Students received different blocks ot 
passages and comprehension questions in 
their booklets according to a procedure 
called “partially biilanced incomplete block 
(pBiB) spiraling.' ’ This procedure assigned 
blocks of questions in a manner that bill- 
anced the positioning of blocks across 
booklets iind balanced the pairing of blocks 
within booklets according to context for 
reading. Blocks were balanced within each 
context for reading and were partially 
baliinced across contexts for reading. The 
spiraling aspect of this procedure cycled the 
booklets for administration so that, typically, 
only a few students in any assessment 
session received the same booklet. 

In addition to the student assessment 
booklets, three other instruments provided 
data relating to the assessment — a teacher 
questionnaire, a school questionniiire, and a 
questionnaire reg;irding students with 
disabilities/limited English proficient 
students (SD/LEP). The teacher question- 
naire w:is administered to teachers of 



fourth- and eighth -grade students participat- 
ing in the assessment and included tour 
sections. The first section focused on the 
teacher's background, the second section on 
instruction, the third section on professioival 
development, and the fourth section on 
standards and assessment. 

The school questionn;iire was given to 
the principal or another administrator in 
each participating school and included 
questions related to school policies, pro- 
grams, and the composition and back- 
ground of the student body. 

The SD/LEP questionn;iire was com- 
pleted by a school staff member knowl- 
edgeable about those students who were 
selected to participate in tlie assessment ^md 
who were identified as having an Individual- 
ized Education Program (lEP) or equivalent 
plan and/or being limited English proficient 
(LEP). ^\n SD/LEP questionnaire was 
completed for each identified student 
regardless of whether the student partici- 
pated in the assessment. Each SD/LEP 
questionnaire asked about the student and 
the special programs in which he or she 
participated. 
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NAEP Samples 

National Sample 

The national results, presented in Tbe 
Nation's KeporC Car'd: Reading 2002, are based 
on nationally representative probability 
samples of fourth-, eighth-, iind twelfth- 
grade students. At grades 4 and 8, the 
national sample in 2002 was a subset of the 
combined S 2 imple of students assessed in 
each participating state, plus an additional 
sample from the states that did not partici- 
pate in the state assessment imd a pnvate 
school sample. In accordance with the 
iSlAEP legislation, the program uses a 
random selection process in order to obtain 
a representative sample of students for 
reporting national and state or jurisdiction 
results. 

Each selected school that participated in 
the assessment and each student assessed 
represents a portion of the population of 
interest. Sampling weights are needed to 
make valid inferences between the student 
samples and the respective populations 
from which they were drawm. Sampling 
weights applied to the national and state 



samples account for disproportionate 
representation due to the oversampling of 
students who attend nonpublic schools and 
schools with high concentrations of Black 
and/or Hispanic students. ^\mong other 
uses, sampling weights also account for 
lower sampling rates for very small schools 
and are used to adjust for school and 
student nonresponse.^ Appropriate sam- 
pling weights were applied to the trial urban 
district samples. 

Testing accommodations (e.g, extended 
time, small group testing were permitted 
for special-needs students selected to 
participate in the NAEP reading assess- 
ments. NAEP inclusion rules were applied, 
and accommodations were offered when a 
student had an Individualized Education 
Program (lEP) because of a disability, was 
protected under Section 504 of the Reha- 
bilitation Act of 1973 because of disability, 
;md/or was identified as being a limited 
English proficient student (LEP); all other 
students were asked to participate in the 
assessment under standard conditions.^ 



^ Additional details regatdiiig the design and stnicture ot tlie national jmd state samples will be tbimd in the technical 
docniueutation section of the NAEP web site at http://uces.ed.gov/uatiousrcportcaEd. 

^ Section 504 of the Rehabilitation Act of 1973 is a civil lights law designed to prohibit discrimination on the basis of 
disability in programs and activities, iuchiding education, that receive fedend dnancijil assistance. 
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Standards for Sample 
Participation and Reporting 
of Results 

In carrying out the 2002 state and Ttrial 
Urbiin District Assessments, the National 
Center tor Education Statistics (NCES) 
established participation rate standards that 
jurisdictions were required to meet in order 
for their results to be reported. NCES 
also established additional standards that 



required the annotation of published results 
for jurisdictions whose sample pvirticipation 
rates were low enough to raise concerns 
about tlieir representativeness. The NCES 
guidelines used to report results in the state 
assessments, and the guidelines for notation 
when there is some risk of nonresponse 
bias in the reported results, iU*e presented in 
tliis section. These guidelines also applied to 
the Trial Urban District Assessments. 



Guideline I 



The publicotion of NAEP results | 

The conditions that will result In the publication of a jurisdiction's results are presented below. 

Guideline 1 ^ Publicotion of Public School Results 

A jurisdiction will hove its public school results published in the NAEP 2002 reading report card (or in other reports that include all state- 
level results) if and only if its weighted participation rate for the initial sample of public schools is greater than or equal to 70 percent. 
Similarly, a jurisdiction will receive a separote NAEP State Report if and only if its weighted participation rate for the initial sample of public 
schools is greater than or equal to 70 percent. 

Discussion: If a jurisdiction's public school participation rate for the initial sample of schools is below 70 percent, there is a substantial 
possibility that bias will be introduced into the assessment results. This possibility remains even after making statistical adjustments to 
compensate for school nonparticipation. There remains the likelihood that, in aggregate, the substitute schools are sufficiently dissimilar 
from the originals they are replacing and represent too great a proportion of the population to discount such a difference. Similarly, the 
assumptions underlying the use of statistical adjustments to compensate for nonparticipation are likely to be significantly violated if the 
initial response rate falls below the 70 percent level. Guideline I takes this into consideration. This guideline is congruent with current 
NAGB policy, which requires that data for jurisdictions that do not have a 70 percent before-substitution participation rate be reported "in a 
different format," and with the Education Information Advisory Committee (EiAC) resolution, which calls for data from such jurisdictions not 
to be published. 



The following guidelines concerning 
school and student participation rates in the 
NiVEP state assessment program were 
established to address four significant ways 
in which nonresponse bias could be intro- 
duced into the jurisdiction sample estimates. 
The four significant ways are overall school 
nonresponse, strata-specific school 



nonresponse, overall student nonresponse, 
and strata-specific student nonresponse. 
Presented on the following p;iges are the 
conditions that will result in a jurisdiction's 
receiving a notation in the 2002 reports. 
Note that in order for a jurisdiction's results 
to be published with no notations, that 
jurisdiction must satisfy ^ill guidelines. 
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Guideline 2 



Reporting school ond student portidpotion rotes with possible bios due to school nonresponse 

Guideline 2 * Nototron for Overall Public School Portidpotion Rote 

A jurisdiction that meets Guideline 1 will receive a notation if its weighted participation rate for the initial sample of public schools was 
below 85 percent and the weighted public school participation rate after substitution was below 90 percent. 

Discussion: For jurisdictions that did not use substitute schools, the participation rates are based on participating schools from the original 
sample. In these situations, the NCES standards specify weighted school participation rates of ot least 85 percent to guard against potential 
bias due to school nonresponse. Thus the first part of these guidelines, referring to the weighted school participation rate for the initial 
sample of schools, is In direct accordance with NCES standards. 

To help ensure adequate sample representation for each jurisdiction participating in the NAEP 2002 state assessments, NAEP provided 
substitutes for nonparticipating public schools. For jurisdictions that used substitute schools, the assessment results will be based on the 
student data from all schools participating from both the original sample and the list of substitutes (unless both an initial school and its 
substitute eventually participated, in which case only the data from the initial school will be used). 

The NCES standards do not explicitly address the use of substitute schools to replace initially selected schools that decide not to participate 
in the assessment. However, considerable technical consideration was given to this issue. Even though the characteristics of the substitute 
schools were matched as closely as possible to the characteristics of the initially selected schools, substitution does not entirely eliminate 
bias due to the nonparticipation of initially selected schools. Thus, for the weighted school participation rotes including substitute schools, 
the guidelines were set at 90 percent. 

if a jurisdiction meets either standard (i.e., 85 percent or higher prior to substitution or 90 percent or higher after substitution), there will 
be no notation for the relevant overall school participation rate. 



Guideline 3 



Important segments of the jurisdiction's student population thot must be adequately represented 
to ovoid possible nonresponse bios 

Guideline 3 * Notation for Strota-Specific Public School Partidpatlon Rates 

A jurisdiction that is not already receiving a notation under Guideline 2 will receive a notation if the somple of public schools included a 
class of schools with similar characteristics that had a weighted participation rate (after substitution) of below 80 percent, ond from which 
the nonparticipating schools together accounted for more than 5 percent of the jurisdiction's total weighted sample of public schools. The 
classes of schools from each of which a jurisdiction needed minimum school portkipation levels were determined by degree of urbonizo- 
tion, minority enrollment, and median household income of the areo in which the school is located. 

Discussion: The NCES standards specify that attention should be given to the representativeness of the sample coverage. Thus, if some 
important segment of the jurisdiction's population is not adequately represented, it Is of concern, regardless of the overoll participation 
rate. 

If nonparticipating schools are concentrated within a particular class of schools, the potential for substantial bias remains, even if the 
overall level of school participation appears to be satisfactory. Nonresponse adjustment cells for public schools hove been formed within 
each jurisdiction, and the schools within each cell are similar with respect to minority enrollment, degree of urbonizotion, and/or median 
household income, as appropriate for each jurisdiction. 

If the weighted response rate, after substitution, for a single adjustment cell falls below 80 percent, and more thon 5 percent (weighted) of 
the sampled schools are nonparticipants from such a cell, the potential for nonresponse bias is too great. This guideline is based on the 
NCES standard for stratum-specific school response rates. 
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Guideline 4 



Possible student nonresponse bios ! 

Guideline 4 - Notation for Overoil Student Portkipation Rate in Public Schools 

A jurisdiction that meets Guideline I will receive o notation if the weighted student response rote within participating public schools was 
below 85 percent. 

Discussion: This guideline follows the NCES stondord of 85 percent for overoll student portkipation rates. The weighted student 
porticipotion rate is hosed on oil eligible students from initioily selected or substitute schools who porticipoted in the ossessment in either an 
initiol session or a moke<up session. If the rote foils below 85 percent, the potential for bios due to students' nonresponse is too great. 



Guideline 5 



Possible nonresponse bios from inadequately represented strata 

Guideline S - Nototlon for Strato-Spedfic Student Participation Rates in Public Schools 

A jurisdiction thot is not already receiving a nototion under Guideline 4 will receive o nototlon if the sampled students within participating 
public schools included a class of students with similor dioracteristics that had a weighted student response rate of below 80 percent, and 
from which the nonresponding students together occounted for more than 5 percent of the jurisdiction's weighted assessable public school 
student somple. Student groups from which a jurisdiction needed minimum levels of participation were determined by the age of the 
student, whether or not the student was classified as a student with a disability (SO) or limited English proficient (LEP), and the type of 
ossessment session, os well os school level of urbonizotion, minority enrollment, and median household income of the area in which the 
school is locoted. 

Discussion: This guideline oddresses the foct thot if nonparticipating students are concentrated within a particular class of students, the 
potentiol for substontiol bios remoins, even if the overoll student participation level appears to be satisfactory. Student nonresponse 
odjustment ceils hove been formed using the school-level nonresponse adjustment cells, together with the student's age and the nature of 
the ossessment session. 

If the weighted response rote for a single odjustment ceil falls below 80 percent, and more than 5 percent (weighted) of the invited 
students who do not porticipote in the ossessment are from such a ceil, the potential for nonresponse bias is too great. This guideline is 
bosed on the NCES stondord for strotum-specific student response rates. 



In the 2002 Trial Urban District. Assess- 
ment, New York City did not meet the 
initial public-school participation rate of 70 
percent at eighth grade. Consequently the 
performance results for this grade were not 
reported. 



At grade 4, New York City also did not 
meet the second guideline (i.e., the weighted 
participation rate for the initial sample of 
schools w^as below 85 percent and the 
weighted school p'^irticipation rate after 
substitution w^as below 90 percent). Results 
for New York City at grade 4 are shown 
with a notation indicating possible bias 
related to nonresponse. 
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Trial Urban District Samples 

Siompling for the Triiil Urbaii District 
Assessment was modeled on NAEP’s state 
sampling procedures. However, school 
substitution was not an option in the Trial 
Urban District Assessment. Tables C.3 and 
C.4 provide a summaty of the 2002 urban 
district school and student participation 
rates for the reading assessment sample. 

The first rate in each table is the weighted 
percentage of schools participating in the 
assessment. This rate is based only on the 
number of schools that were initially se- 
lected for tlie assessment. The numerator of 
this rate is the sum of the number of 
students represented by each selected school 
that participated in the assessment. The 
denominator is the sum of the number of 
students represented by each of the selected 
schools that had eligible students enrolled. 

Also presented in tables C.3 ;md C.4 are 
weighted student participation rates. The 
numerator of this rate is die estimated 



number of students that each student 
represents across all students assessed (in 
either an initial session or a makeup ses- 
sion). The denominator of this rate is the 
estimated number of students represented 
in die sample, across all eligible sampled 
students in participating schools. The 
number ot students that each student 
represents is mainly determined by the 
probabiliy that a student is included in the 
sample, with necessary adjustments made 
for other factors. The overall participation 
rates take into account the weighted per- 
centage ot school participation and the 
weighted percentage of student participa- 
tion after makeup sessions. 

For the grade 8 sample, New York City’s 
school and student response rates did not 
meet NCES standards. Consequently the 
performance results for this grade were not 
reported. 
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Toble C.3 Weighted school and student porticipotion rates, grade 4: By urban district, 2002 





Number 
of schools 


Number 
of students 


School rate 


Student 

rate 


Overall 

rate 


Atlonto 


49 


1,509 


98 


93 


92 


Chicogo 


76 


2,100 


95 


92 


87 


District of Columbio 


117 


2,810 


100 


90 


90 


Houston 


49 


1,326 


98 


95 


93 


Los Angeles 


76 


2,017 


100 


93 


93 


New York City ^ 


38 


947 


76 


89 


67 



^ Although deemed sufficient for reporting, the torgei response rote specified in the tMEP guidelines was not met. 

SOURCE: U.S. Deportment of &fucation, Institute of Education Sciences, Notionol C«iter for Bfucotion Statistics, Notionol Assessment of Educational Progress (NAEP), 2002 Trial Urhon Distnct Reading 
Assessment. 

Table C.4 Weighted school ond student porticipotion rotes, grade 8: By urbon district, 2002 





Number 
of schools 


Number 
of students 


School rate 


Student 

rate 


Overall 

rate 


Atlonto 


15 


1,281 


100 


91 


91 


Chicago 


69 


1,562 


94 


92 


86 


District of Columbio 


36 


1,795 


100 


85 


85 


Houston 


34 


1,110 


96 


89 


86 


Los Angeles 


66 


1,778 


97 


90 


87 


New York Gty ^ 


31 


692 


63 


81 


51 



^ Indicates that the dtstrict did not meet the guideime fm 70 percent school portkipotion in 2002. 

SOURCE: U.S. Department of Education, Institute of Educotion Sciences, National Center for Mjcotlcm Statistics, Notional Assessment of Educational Progress (NAEP), 2002 Trial Urban Distnct Reading 
Assessment. 



Results from the 2002 reading assess- 
ments are reported (on a trial basis) here on 
district-level samples of fourth- and ei^th- 
grade students in the large urbiin school 
districts that participated in the Trial Urban 
District Assessment and in tlie District of 
Columbia. The samples of students in the 
urban school districts represent augmenta- 
tions of students who would norm:illy be 
selected as part of state samples. These 



samples allow reliable subgroup reporting in 
these districts. All students at ‘"lower” 
geographical levels are assumed to be part 
of ""higher-level” s'^imples. For exiimple, 
Houston is one of the urb^m districts 
included in the Trial Urban District Assess- 
ment. Data from students tested in the 
Houston sample are used to report results 
for Houston, and also contribute to the 
'lexas and the national estimates. 
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Tables C.5 and C.6 display the target 
student and school sample sizes planned for 
the Trial Urban District Assessment. The 
first column contains the phinned number 
of schools for each district. The second 



column contains the number of schools that 
would have been sampled by Nj\EP in each 
district had tliere been no Trial Urban 
District Assessment. The last column shows 
the planned sample size. 



Table CS Number of schools and students planned for the Trial Urban District Assessment grade 4: 
By urban district 2002 



Grade 4 


Number of schools 


Number of schools 


1 


Atlanto 
Chicoga 
Houston 
Los Angeles 
New York City 


for trial assessment 

1 51 

1 31 

i 51 

76 

♦ 52 


normally sampled in NAEP 
6 

24 

5 

12 

41 


Number of students 

1250 

1900 

1250 

1900 

1250 


t Although deemed sufficient for reporting, the target response rote specif^ in 


the NAEP guidelines was not met. 




SOURCE: U.S. Deportment of Edutotion, Institute of Education Sciences, Notionol Center for Education Statistics, Notionol Assessment of Educotionol Progress (NAEP), 2002 Trial Urban District Reading 
Assessment. 


Table C*6 Number of schools and students planned for the Trial Urbon District Assessment, grode 8: 
By urban district, 2002 


Grade 8 


Number of schools 


Number of schools 


1 




for trial assessment 


normally sampled in NAEP 


Number of students 


Atlonta 


17 


8 


1250 


Chicago 


79 


26 


1900 


Houston 


39 


6 


1250 


Los Angeles 


68 


13 


1900 


Hew York City 


♦ 52 


37 


1250 



n in 2002. 

SOURCE: U.S. Department of ^lutotion, institute of Edutotion Scientos, Notionol Center for Educotion Stothtics, National Assnsment of Edutotionol Progress (NAEP), 2002 Trid Urbon District Reading 
Assessment. 



Note that the sample sizes vary among 
districts. The study was designed to allow 
the examination of the quality of data that 
resulted from samples of different sizes. 



Therefore, huger samples were selected in 
two of the districts (Chicago and Los 
xVngeles), and sniioller samples in the remain- 
ing three. 
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Data Collection and Scoring 

The 2002 reading assessment, was con- 
ducted from January to M:u*ch 2002. Data 
collection for the 2002 assessment at the 
national, state and distnct levels was con- 
ducted by trained field staff from Westat. 

Materials from the 2002 assessment were 
shipped to Pearson, where trained staff 
evaluated the responses to tlie constructed- 
response questions using scoring rubrics or 
guides prep;ired by Education;il Testing 
Service (ETS). Each constructed-response 
question had a unique scoring guide that 
defined the criteria used to evaluate stu- 
dents’ responses. The extended constructed- 
response questions were evaluated with 
four-level guides, arid almost all of the short 
constructed-response questions were rated 
accordiiig to three-level guides that permit- 
ted partial credit. Other short constructed- 
response questions were scored as either 
acceptable or unacceptable. 

For the 2002 reading assessment, 
4,023,861 constructed responses were 
scored. This number includes rescoring to 
monitor in ter rater reliability. The within - 
year average percentage of exact agreement 
for the 2002 natioiial reliability sample was 
92 percent at fourth grade and 91 percent at 
eighth grade. 

Data Analysis and IRT Scaling 

Subsequent to the professional scoring, all 
information was transcribed into the N^AEP 
database at ETS. Each processing activity 
was conducted with rigorous quality control. 
After the assessment information was 
compiled in the database, the data were 
weighted according to the population 



structure. The weighting for the national, 
state and trial urban district samples re- 
flected the probability of selection for each 
student as a result of the sampling design, 
adjusted for nonresponse.*^ 

Analyses were then conducted to deter- 
mine the percentages of students who gave 
various responses to each cognitive and 
background question. In determining these 
percentages for the cognitive questions, a 
distinction was made between missing 
responses at the end of a block (i.e., missing 
responses subsequent to the last question 
the student answered) and missing re- 
sponses prior to the last observed response. 
A'lissing responses before the last observed 
response were considered intentional 
omissions. In analysis, omitted responses to 
multiple-choice items were scored as frac- 
tionally correct.^ For constructed-response 
items, omitted responses were placed into 
the lowest score category. Missing responses 
at the end of the block were considered 
‘hot reached” and treated as if the ques- 
tions had not been presented to the student. 
In calculating response percentages for each 
question, only students classified as having 
been presented the question were included 
in the denominator of the statistic. 

It is standiird Ni\EP practice to treat ;ill 
nonrespondents to the last question in a 
block as if they had not reached the ques- 
tion. For multiple-choice and short con- 
staicted-response questions, this practice 
produces a reasonable pattern of results in 
that the proportion reaching the last ques- 
tion is not dramatically smaller than the 
proportion reaching the next-to-last ques- 



^ Wfeighriug procedures nre described more biJly in die “Weighting and Variance Estimation"" section later in this 
document Additional information about the use of weighting procedures will lie fbimd in the technical documeuta- 
lion section of tlie NAEP web site at http:/ / nces.ed.gov/naliousreportcarcJ. 

^ Lord, F. M. (1980). Applications of Item ^sponse Theory to Practical Testing Problems. Hillsdale, NJ: Lawrence Erlbaum 
Associates. 
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tioii. However, for reading blocks that 
ended with extended constructed-response 
questions, the standard practice could result 
in extremely large drops in the proportion 
of students attempting some of the final 
questions. Therefore, for blocks ending with 
an extended constructed-response question, 
students who answered the next-to-last 
question but did not respond to the ex- 
tended constructed-response question were 
classified as having intentionally omitted the 
last question. 

Item Response Theory (IRT) was used to 
estimate average reading scale scores for the 
nation and for various subgroups of interest 
within the nation. IRT models the probabil- 
ity of answering a c|uestion in a certain way 
as a mathematical function of proficiency or 
skill. Nk\EP used IRT analysis to provide a 
common scale on which perform:ince can 
be compared among groups such as those 
defined by characteristics, including gender 
and race/ethnicity. 

The results for 1992, 1994, 1998, 2000 
and 2002 are presented on the NAEP 
reading scales in the Nation's Keport Card: 
Reading 2002. In 1992, a scale ranging from 
0 to 500 was created to report performance 
for each reading purpose — literary ;ind 
information at grade 4, and literary, infor- 
mation, and task at grades 8 and 12. The 
scales summiirize student performance 
across all three types of questions in the 
assessment (multiple-choice, short con- 
structed-response, and extended con- 
structed-response). Results from subsequent 
reading assessments (1994, 1998, 2000, and 
2002) are reported on these scales. 



Each reading scale was initi;illy based on 
the distribution of student performance 
across all three grades in the 1992 national 
assessment (grades 4, 8, and 12). In that 
year, the scales had an average of 250 and a 
standard deviation of 50. In addition, a 
composite sc;ile was created as an overall 
measure of students’ reading performance. 
This composite sc;ile is a weighted average 
of the three separate scales for the three 
reading purposes at grade 8 and two reading 
purposes at grade 4. The weight for each 
reading purpose is proportional to the 
relative importance assigned to the reading 
purpose by the specifications developed 
through the consensus phinning process and 
given in the framework. 

In producing the reading scales, three 
distinct IRT models were used. Multiple- 
choice questions were scaled using the 
three-parameter logistic (3PL) model; short 
constructed-response questions rated as 
acceptable or unacceptable were scaled 
using the two-parameter logistic (2PL) 
model; and short constructed-response 
questions rated according to a three-level 
scoring guide, as well as extended con- 
structed-response questions rated on a 
four-level scoring guide, were scaled using 
a Generalised Partial-Credit (GPC) model.^' 
Developed by ETS and first used in 1992, 
the GPC model permits the scaling of 
questions scored according to multipoint 
rating schemes. The model takes full advan- 
tage of the information available from each 
of the student response categories used for 
these more complex constructed-response 
questions.' 



^ Murjiki, E. (1992). A Geuetalizetl Partial Credit Model: AppUcatiou of au EM Algorithm. Applied P^'chohpcal 
Measuremmty 16ig)^ 159—176. 

~ More detailed iixlbrmatiou regarding llie IRT luiulyses used in Nx*VEP will be found in tlie technical documeiualion 
section of die NAEP web site at http:/ /nces.ed.gov/nationsreportcard. 
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The reading scale is composed of three 
types of questions: multiple-choice, short 
constructed-response (scored either di- 
chotomously or allowing for pjurtial credit), 
and extended constructed-response (scored 
according to a partial-credit model). Unfor- 
tunately, the question of how much infor- 
mation different t)q)es ot questions contrib- 
ute to the reading scale has no simple 
answer. The information provided by a 
given question is determined by the IRT 
model used to scale the question. It is a 
function of the item parameters and varies 
by level of reading proficiency.^ Thus, the 
answer to the query 'TIow much informa- 
tion do the different types of questions 
provide?’’ will differ for each level of 
reading performance. WTien considering the 
composite reading scale, the answer is even 
more complicated. The reading data are 
scaled separately by the two purposes for 
reading (reading for information and read- 
ing for literary experience) for grade 4, and 
the three purposes for reading (reading for 
information, reading for literary experience, 
and reading to perform a task) for grade 8, 
resulting in two or three separate sub scales 
at each grade. The composite scale is a 
weighted combination of these subscales. 
IRT information functions are only strictly 
comparable when the item parameters are 
estimated together. Because the composite 
scale is based on two or three separate 
estimation runs, there is no direct way to 
compare the information provided by the 
questions on the composite scale. 



Because of the assessment booklet 
spiraling design used by NAEP, students do 
not receive enough questions about a 
specific topic to provide reliable informa- 
tion about individual performance. (For 
more information on the assessment book- 
let, see ‘^The Assessment Design” section 
presented earlier in this appendix.) Tradi- 
tional test scores for individual students, 
even those based on IRT, would result in 
misleading estimates of population ch^irac- 
teristics, such as subgroup means and 
percentages ot students at or above a 
certain scale -score level. However, it is 
N^AEP’s goal to estimate these population 
characteristics. As discussed by Mislevy and 
Sheehan (1987), N^AEP’s objectives can be 
achieved with methodologies that produce 
estimates of the population-level parameters 
directly, without the intermediary computa- 
tion of estimates of individuals.^ This is 
accomplished using marginal estimation 
scaling model techniques tor latent variables. 
Under the assumptions of the scaling 
models, these population estimates will be 
consistent in the sense that the estimates 
approach the model-based population 
values as the sample size increases. This 
would not be the case for population 
estimates obtained by aggregating optimal 
estimates of individual performance. 

Weighting and Variance 
Estimation 

A complex sampling design was used to 
select the students who were assessed. 

The properties of a sample selected through 
such a design may be very different from 



^ DonogJiiie, J. R. (1994). An Empirical Exaiiiinntioii ot the IRV Inibrmation of Polyroinoiisly Scomd Reading Items 
Under the Geuerali2ed Partial Credit Model. Journal of Educationa/ Measuremeut^ 29.S— 31 1. 

Mislew, R J-, Slieelian, K. M. (1987). Marginal Estimation Procedures. In A. E. Beaton (Ed.) ItnpiemerMng the 
Nen' Des(s>u: TheN/^iEP 1983-1984 Technical Keport No. 15-TR-20), pp. 293-260. Princeton, NJ; Educational 
Testing Service. 

For dieoreiical luid empirical juslilicadon of die procedures employed, see Mislevy, R.J. (1988) Randoinhsaiion-Based 
Inferences About Latent Variables from Complex Samples. Pcychometrika^ 36(2), 177-196. 
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tliose of a simple random sample, in which 
every stxident in die target population has an 
equal chance of selection and in which the 
observations from different sampled stu- 
dents can be considered to be statistically 
independent of one ^mother. Therefore, the 
properties of the sample for the data 
collection design were taken into account 
during the analysis of the assessment data. 

One way that the properties of the 
sample design were addressed was by using 
sampling weights to account tor the fact 
diat the probabilities of selection were not 
identical for ;ill students. All population and 
subpopulation characteristics based on the 
assessment data were estimated using 
sampling weights. These weights included 
adjustments for school and student 
nonresponse. 

Not only must appropriate estimates of 
population characteristics be derived, but 
appropriate measures of the degree of 
uncertainty must be obtained for those 
statistics. Two components of uncertainty 
are accounted for in the variability of 
statistics based on student ability: 1) the 
uncertainty due to sampling only a relatively 
sm'all number ot students, and 2) the 
uncertainty due to sampling only a portion 
of the cognitive domain of interest. The 
first component account:s for the variability^ 
associated with the estimated percentages 
of students who had certain background 
characteristics or who answered a certain 
cognitive question correctly. 



Because NAEP uses complex sampling 
procedures, conventional formulas for 
estimating sampling variability that assume 
simple random sampling are inappropriate. 
NAEP uses a jackknife replication proce- 
dure to estimate standard errors. The 
jackknife standard error provides a reason- 
able measure of uncertainty for any student 
information that can be observed witliout 
error. However, because each student 
typically responds to only a few questions 
within any context of reading, the scale 
score for any single student would be 
imprecise. In this case, NAEP's marginal 
estimation methodology can be used to 
describe the performance of groups and 
subgroups of students. The estimate of the 
variiince of the students' posterior scale 
score distributions (which reflect the impre- 
cision due to lack of measurement accuracy) 
is computed. This component of variability 
is then included in the standard errors of 
N^\EP scale scores.'^ 

Typically, when the standard error is 
based on a small number of students or 
when the group of students is enrolled in a 
small number of schools, the amount of 
uncertainty associated with the estimation 
of standard errors may be quite large. 
Additional details concerning procedures 
for identitying such standard errors will be 
found in the technical documentation 
section of the N^AEP web site at http:// 
nces.ed.gov/nationsreportcard. 



For iiiilher deuiis, see Joluisou, E. G., and Rust, K. F (1992). Population Inferences and Variance Estixuadou for 
NAEP Data. Journal of Eiir/Mtiona/ Sfaris/ic's^ 77(2), 175—190. 
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The reader is reminded that, as with 
findings from all surveys, N/VEP results are 
subject to other kinds of error, including 
the effects of imperfect adjustment for 
student and school nonresponse and un- 
knowable effects associated widi die par- 
ticular instrumentation and data collection 
methods. Nonsampling errors can be 
attributed to a number of sources — inability 
to obtain complete information about all 
selected schools in the sample (some stu- 
dents or schools refiised to participate, or 
students participated but answered only 
certiiiri questions); ambiguous definitions; 
differences in interpreting questions; inabil- 
ity or unwillingness to give correct back- 
ground intormation; mistakes in recording, 
coding, or scoring data; and other errors in 
collecting, processing, sampling, and esti- 
mating missing data. The extent of 
nonsampling errors is difficult to estimate 
and, because of dieir nature, the impact of 
such errors cannot be reflected in die data- 
based estimates of uncertainty provided in 
N^\EP reports. 



Drawing Inferences 
from the Results 

The reported statistics are estimates and 
are therefore subject to a measure of 
uncertainty. Tliere are two sources of such 
uncertainty. First, N^\EP uses a sample of 
students rather than testing all students. 
Second, all assessments have some amount 
of uncertainty related to the fact that they 
cannot ask all questions that might be asked 
in a content area. The magnitude of this 
uncertainty is reflected in the estimated 
standard error of each of die estimates. 
Wlien die percentages or average scale 
scores of certain groups arc compared, the 
estimated standard error should be taken 
into account, iind observed simiLuaties or 
differences should not be relied on solely. 
Therefore, the compiu'isons are b;ised on 
statistical tests that consider the estimated 
standard errors of those statistics and the 
magnitude of the difference among the 
averages or percentages. 

For the data presented in this report, all 
the estimates have corresponding estimated 
standard errors. For example, table C.7 
shows the average scale score for the NAEP 
2002 Trial Urban District Assessment and 
percentages of students at or above achieve- 
ment levels by gender for grade 4. Also, 
table C.8 shows the scores and standard 
errors for the 25th, 50th, and 75th percen- 
tiles at grades 4 and 8. Estimated standard 
errors appear in parentheses next to each 
estimated scale score or percentage. For the 
estimated standard errors corresponding to 
other data in this report, the reader can 
consult the NCES web site at http:/ / 
nces.ed.gov/ nationsreportcard/ naepdata/. 
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Toble C7 Average reodtng scole scores, percentoge of students ot or obove each achievement level, 
and estimated standard errors, by gender, grode 4 public schools: By urbon district, 2002 



i^ale 



Female 





Percentage 


Average 




At or above 


At or above 






of sfodents 


stole score 


Below Bask 


Bask 


Profitieaf 


Advanted 


Nation (Public) 


51 (0.3) 


214(0,5) 


41 (0,6) 


59 (0,6) 


26(0,5) 


5(0,2) 


Central city (Public) ^ 


50(0.4) 


204(0,7) ** 


53(0,8) ** 


47(0,8)** 


19(0,6) ** 


3(0,3) ** 


Atlanta 


47(1.3) 


191(2,5) 


69(2,6) *'** 


31 (2,6) *'** 


11 (2,3) *'** 


2(1,0) 


Chicago 


50(1.3) 


189(2,0) ’ 


70(2.2) *<** 


30(2,2) *'** 


9(1.1)*'** 


1 (0.4) *'** 


District of Columbia 


49(1.2) 


185(1.2) 


74(1.4) *'** 


26(1.4) *'** 


8(0,9)*'** 


1 (0,4) *'** 


Houston 


51 (1.5) 


204(3.3) 


55 (3,9) ** 


45(3.9) ** 


16(3,4) ** 


3(1,2) 


Los Angeles 


51 (1.2) 


188(2.2)*'** 


70(1.9) *'** 


30(1.9) *'** 


10(1,4) *■** 


1 (0,4) *'** 


New York City ^ 


50(1.5) 


199(2.7)** 


61 (3.9) *'** 


39(3.9) *'** 


14(2,7) ** 


3(0,9) ** 


Notion (Public) 


49(0.3) 


220(0.5) 


35(0.6) 


65 (0.6) 


33(0,6) 


8(0.3) 


Central city (Public) ^ 


50(0.4) 


211(0.7) "* 


45(0.8) ** 


55(0,8) ** 


24(0,9) ** 


5 (0.4) ** 


Atlanta 


53(1.3) 


200(1.7) *'** 


60(2.4) *'** 


40(2,4) *'** 


13(1,4) *'** 


4(0.8) ** 


Chicago 


50(1.3) 


198(2.0) *■** 


62(2.3) *'** 


38(2.3) *'** 


12(1,4) *'** 


2(0,7)*'** 


District of Columbia 


51 (1.2) 


196(1.2) *'** 


64(1.5) *'** 


36(1.5) *'** 


11(1,0) *'** 


2(0,4) *'** 


Houston 


49(1.5) 


208 (2.7) ** 


50(3.1)** 


50(3.1) ** 


19(3.0) ** 


3(1.5) ** 


Los Angeles 


49(1.2) 


194 (2.2) *'** 


64(2.9) *'** 


36(2.9) *'** 


12(1.7) *'** 


2(0,5) *'** 


New York City ^ 


50(1.5) 


213(3.0) ** 


45(3.5) ** 


55(3,5) ** 


23 (3.5) ** 


7(2.5) 



^ AiHiough deemed sufficient for reporting, tfio forget response rote specified in the NAEP guidelines wo$ not n»t. 

* Significantly different from central city public schools. 

'* Significontiy different from notim (public schools}. 

^ For comparison, at fourth grade 65 percent of students in centrol city public schools ond 40 percent in public schools nodonolly were non- White. Also, 61 percent of students in cenlrol city public schools 
ond 43 percent in public schools notionolly were eligible for free/reduced*prlce school lunch. 

NOTE: Standard errors of the estimated percentoges and scale scores appear in parentheses. Percentoges below ond at or above Basi( may not odd to 100, due to rounding. 

SOURCE: U.S. Deportment of Education, Institute of Educotion Sciences, Notional Center for Education Statistics, Notional Assessment of Educational Progress (NAEP}, 2002 Trio! Urban District Reading 
Assessment. 
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Toble C.8 Selected reading percentiles and estimated standard errors, grades 4 and 8 public schools: 
By urban district, 2002 



Scale score distribution 

25th percentile 50th percentile 75th percentile 



Grade 4 



Nation {PobHc} 


194(0.6) 


219(0.4) 


242(0.5) 


Centrol city (Public) ^ 


183(0.8)*" 


209 (0.7) ** 


234(0.7)** 


Atlonta 


171 (2.3) *"'* 


194(1.4)**-* 


219(2.1)**-* 


Chkogo 


170(2.3) **'* 


194(2.1)**-* 


217(1.8) **-* 


District of Columblo 


167(1.3) *** 


191(0.9) **-* 


215(1.3) **-* 


Houston 


183(2.9)** 


206(2.8)** 


229 (3.9)** 


Los Angeles 


165(2.5)**-* 


190(1.3)**-* 


217(2.2) **-* 


New York City ^ 


182(2.3)** 


206 (2.6) ** 


230(3.6)** 


Grade s 


Notion (Public) 


242(0.5) 


265(0.6) 


286(0.5) 


Centrol dty (Public) ^ 


232 (0.9) ** 


256(0.7) ** 


278 (0.8)** 


Atlanta 


214(2.3) **'* 


236(2.6) **-* 


259(1.4)**-* 


Cbicogo 


231(1.8)** 


251 (2.2) **'* 


270(2.6) **-* 


District of Columbio 


219(3.3) **'* 


241(1.3) **-* 


262(1.7) **'* 


Houston 


226 (2.9)** 


251 (2.2) **-* 


273(1.4) **'* 


Los Angeles 


213(2.1)**-* 


238(1.4) **-* 


261(1.4) **-* 



t Abiiough deemed sufficteel for reporting, the target response rote specified in the NAEP gtildeiines was not met. 

' SIgnificontiy differ»it from centrol city pubhc schoois. 

Signifkontly different from nation (pobiic sdtoob). 

^ For comporison, at fourth grode 65 percent of students in central dty public schoois ond 40 percent in pubb'c schoois nationally were non*Wtute. Also, 61 percent of students in central city public schools 
and 43 percent in pubhc schools nationally were eiigitde for fres/reduced^rice school lunch. 

^ For comporison, at eighth grode 61 pwcent of students in central dty public schools ond 36 percent in public schools nationally were non-Wtute. Also, 47 percent of students in centrol city pubbc schools 
ond 34 percent in public schools national were eligitde for free/reduced^rice school lunch. 

SOUKCE: U.S. Deportment of Education, Institute of bfucation Sciences, Notionoi Center for Elucotion Statistics, National Assessnent of Educotionoi Progress (KAEP), 2002 Trial iltbon District Reading 
Assessment. 



Using confidence intervals based on the 
standard errors provides a way to take into 
account the uncertainty associated with 
sample estimates «ind to make inferences 
abc)ut the population averages and percent- 
ages in a manner that reflects that uncer- 
tainty, An estimated sample average scale 
score plus or minus 1.96 standard errors 
approximates a 95-percent confidence 



intetwal for the corresponding population 
quantity. This statement means that one can 
conclude with an approximately 95 percent 
level of confidence that the average perfor- 
mance of the entire population of interest 
(e.g., all fourth-grade students in public 
schools) is within plus or minus 1.96 stan- 
dard errors of the sample average. 
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For example, suppose diat the average 
reading scale score of the students in a 
particular group was 256 with an estimated 
standard error of 1.2. approximately 95 
percent confidence interval for the popula- 
tion quantity would be as follows: 

Average ± 1.96 standard errors 
256 ± 1.96 X 1.2 
256 ± 2.4 
(253.6, 258.4) 

Thus, one can conclude with a 95 percent 
level of confidence that the average scale 
score for the entire population of students 
in that group is between 253.6 and 258.4. It 
should be noted that this example and the 
examples in the following sections are 
illustrative. More precise estimates carried 
out to one or more decimal places lU^e used 
in the actual iinalyses. 

Similar confidence interv.ils can be 
constructed for percentages, if the percent- 
ages are not extremely large or extremely 
small. Extreme percentages should be 
interpreted with caution. Adding or sub- 
tracting the standard errors associated with 
extreme percentages could cause the confi- 
dence interval to exceed 100 percent or fall 
below 0 percent, resulting in numbers’ that 
are not meaningful. A more complete 
discussion of extreme percentages will 
appear in the technical documentation 
section of the NAEP web site at http:/ / 
nces.ed.gov/nationsreportcard. 



Analyzing Group Differences 
in Averages and Percentages 

Statistic*.!! tests determine whether the 
evidence, based on the data from the groups 
in the sample, is strong enough to conclude 
that the averages or percentages are acmally 
ditferent for those groups in the popul*ation. 
If the evidence is strong (i.e., the difference 
IS statistically significant), the report de- 
scribes the group averages or percentages as 
being different (e.g., one group performed 
higher or lower tJian another group), reg:U'd- 
less of whether the sample averages or 
percentages appe:ir to be approximately the 
same. The reader is cautioned to rely on the 
results of the statistical tests rather than on 
the apparent m*agnitude of the difference 
between sample averages or percentages 
when determining whether, the sample 
differences are likely to represent acmal 
differences among the groups in the popula- 
tion. 

To determine whether a real difference 
exists between the average scale scores (or 
percentages of a certain attribute) for two 
groups in the population, one needs to 
obtain an estimate of the degree of uncer- 
tainty associated with the difference be- 
tween the averages (or percentages) of these 
groups for the sample. This estimate of the 
degree of uncertainty, called the ‘'standard 
error of the difference’' between the 
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groups, is obtained hy taking the square of 
each group’s standard error, summing the 
squared standard errors, and taking the 
square root of that sum. 

Standard Error of the Difference = 

= V(SE,^ + SE,^) 

The st'^mdard error of the difference, just 
like the standard error for an individual 
group average or percentage, c;m be used to 
help determine whether differences among 
groups in the population are real. The 
difference between the averages or percent- 
ages of the two groups plus or minus 1.96 
standard errors of the difference represents 
at! approximately 95 percent confidence 
interval. Tf the resulting interval includes 
zero, there is insufficient evidence to claim a 
real difference between the group's in the 
population. If the interval does not contain 
zero, the difference between the groups is 
statistically significant at the 0.05 level. 



The following example of comparing 
groups addresses the problem of determin- 
ing whether the average reading sciile score 
of group A is higher thion that ot group B. 
The sample estimates of the average scale 
scores and estimated standard errors are as 
follows: 



Group 


Average 


Standard 




Scale Score 


Error 


A 


218 


0.9 


B 


216 


1.1 



The difference between the estimates of 
the average scale scores of groups A and B 
is two points (218—216). The estimated 
standiird error of this difference is 

7(0.9‘ + l.T;) = 1.4 

Thus, an approximately 95 percent 
confidence interval for this difference is 
plus or minus 1.96 standard errors of the 
difference. 

2 ± 1.96 X 1.4 
2 ± 2.7 
(-0.7, 4.7) 

The value zero is within the confidence 
interval; therefore, there is insufficient 
evidence to claim that group A outper- 
formed group B. 
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Conducting Multiple Tests 

The procedures in the previous section and 
the certainty ascribed to intervals (e.g., a 95 
percent confidence interval) are based on 
statistical theory that assumes that only one 
confidence interval or test of statistical 
significance is being calculated. However, 
there are times when many different groups 
are being compared (i.e., multiple sets of 
confidence intervals are being analyzed). In 
sets of confidence intervals, statistical 
theory indicates that the certainty associated 
with the entire set of intervals is less thaii 
tliat attributable to each individual compari- 
son from the set. To hold the significance 
level for the set of comparisons at a particu- 
lar level (e.g., 0.05), adjustments (called 
‘"multiple comparison procedures”) must be 
made to the methods described in the 
previous section.^' One such procedure, the 
Benjamini-Hochberg False Discovery Riite 
(FDR) procedure was used to control the 
certainty level. 

Unlike the other multiple comparison 
procedures that control die familywise error 
rate (i.e., the probability of making even one 



false rejection in the set of comparisons), 
the FDR procedure controls the expected 
proportion of falsely rejected hypotheses. 
Furthermore, the FDR procedure used in 
N-;\EP is considered appropriately less 
conservative than familywise procedures for 
large families of comparisons. 

Therefore, the FDR procedure is more 
suitable for multiple comparisons in NAEP 
than other procedures. A detailed descrip- 
tion of the FDR procedure will appear in 
the technical documentation section of the 
NAEP web site at http:/ /nces.ed.gov/ 
nationsreportcard. 

To illustrate how the FDR procedure is 
used, consider the comparisons of current 
and previous years’ average reading scale 
scores for the five groups presented in table 
C.9. Note that the difference in average 
scale scores ;ind the estimated st;indard 
error of the difference are calculated in a 
way comparable with that ot the example in 
the previous section. The test statistic 
shown is the difference in average scale 
scores divided by the estimated standvird 
error of the difference. 



Table C.9 Exomple of FDR comporisons of overage scale scores for different groups of students 





Previous 


yeor 


Current yeor 


Previous year and current yeor 


1 




Average 


Standard 


Average 


Standard 


Difference 


Standard 


Test 


Percent 




scale score 


error 


scale score 


error 


in averages 


errar of difference 


stotistic 


confidence^ 


Group 1 


224 


1.3 


226 


1.0 


2.08 


1.62 


1.29 


20 


Group 2 


187 


1.7 


193 


1.7 


6.31 


2.36 


2.68 


1 


Group 3 


191 


2.6 


197 


1.7 


6.63 


3.08 


2.15 


4 


Group 4 


229 


4.4 


232 


4.6 


3.24 


6.35 


0.51 


62 


Group 5 


201 


3.4 


196 


4.7 


-5.51 


5.81 


-0.95 


35 



^ Use percent confidence H 2(1 'F{x)) where F(x) is the cumulative dntribution of the t'cHstribution wn)h the degrees of freedom odjusted lo reflect the comfdexilies of the sample design. 




Miller, R. G. (1966). Simultaneous Stadstieal Injh'euce. New York: Jolui Wiley & Sous, Inc. 

Benjjumni, Y., and Hochberg, Y (1993). Coutrolliug tlie False Discovery Rate: A Practical and Powerful Approach to 
Multiple Testing. Journal of the Ri^l Statistical Socie^'. Series B, no. 1, 289—.300. 

Williiuus, y S. L., Jones, L. V., andd iikey, J. W (1999). Conrrolling Error in Multiple Comparisons witli Examples 
From Statc-to-Statc Difletcnces in Educational Acliievemenf. Jormtal of Educational and Behavioral Statistics^ -'^(1), 42-69. 
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The difference in average scale scores ;md 
the estimated standard error of that differ- 
ence can be used to find an approximately 
95 percent confidence interval, as in the 
example in the previous section, or they can 
be used to identify a confidence percentage. 
In the example in the previous section, 
because an approximately 95 percent 
confidence interval was desired, the number 
1.96 was used to multiply the estimated 
standiird error of the difference to create 
tlie approximate confidence interval. In the 
current example, the confidence interv'^al tor 
the test statistics is identified from statistical 
tables. Instead of checking to see if zero is 
within the 95 percent confidence interval 
about the meiin, the significance level from 
the statistical tables can be directly com- 
pared to 100 - 95 — 5 percent. 

If the comparison of average scale scores 
across two years were made for only one of 
the five groups, there would be a significant 
difference between the average scale scores 
for the two years if the significance level 
were less than 5 percent. However, because 
we are interested in the difference in average 
scale scores across the tu^o yeiirs for all five 
of the groups, comparing each of the 
significance levels to 5 percent is not ad- 
equate. Groups of students defined by 
shared characteristics, such as racial/ethnic 
groups, are treated as sets or families when 
making comparisons. However, compari- 
sons of average scale scores for each pair of 
years were treated sepiirately, so the steps 
described in this example would be repli- 
cated for the comparison of other current 
and previous year average scale scores. 



Using the FDR procedure to take into 
account that all comparisons are of interest 
to us, the percents of confidence in the 
example are ordered from largest to small- 
est: 62, 35, 20, 4, and 1. In the FDR proce- 
dure, 62 percent confidence for the group 4 
comparison would be compared to 5 per- 
cent, 35 percent for the group 5 comparison 
would be compared to 0.05 X (5— 1)/5 = 
0.04 = 4 percent, 20 percent for the 
group 1 comparison would be compared to 
0.05 X (5— 2)/5 = 0.03 = 3 percent, 4 
percent for the group 3 comparison would 
be compared to 0.05 X (5“^3)/5 0.02 = 

2 percent, vmd 1 percent for the group 2 
comparison (actually slightly smaller than 1 
prior to rounding would be compared to 
0.05 X (5-4)/5 - 0.01 = 1 percent. The 
procedure stops with die first contrast 
found to be signific'ant. The last of these 
comparisons is the only one for which the 
percent confidence is srnviller than the FDR 
procedure value. The difference in the 
current year and previous years" average 
scale scores tor the group 2 students is 
significant; for all ot the other groups, 
average scale scores for current and previ- 
ous year are not significantly different from 
one another. In practice, a very small num- 
ber of counterintuitive results occur when 
the FDR procedures are used to examine 
between-yeiir differences in subgroup 
results by jurisdiction. In those cases, results 
were not included in this report. NCES is 
continuing to evaluate the use of FDR and 
multiple-comparison procedures for future 
reporting. 



15 



*llie level of confirfeuce times the niiiuber of comparisons minus one divided by tlic iimnber of comparisous is 
0.05X(5” l)/5 = 0.04 = 4 percent. 
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NAEP Reporting Groups 

Results are provided for groups of 
students defined by shared characteristics — 
gender, race /ethnicity, school’s type of 
location, and eligibility for free/reduced- 
price school lunch. Based on participation 
rate criteria, results are reported for sub- 
populations only when sufficient numbers 
of students and adequate school representa- 
tion- are present. The minimum requirement 
is at least 62 students in a particular sub- 
group from at least five primary sampling 
units (PSUs).^^ The first-stage sampling 
units in the selection of Trial Urban District 
Assessment samples are schools. However, 
tlie dal:a for all students, regardless ot 
whether their subgroup was reported 
separately, were included in computing 
overall results. Definitions of the subpopu- 
lations are presented below. Note that not 
all of the reporting groups used for the 
national report card are included in this 
report on the urban districts. 

Gender 

Results are reported separately for males 
and females. 

Roce/Efhnicify 

In all NAEP assessments, data about 
student race/ethnicity is collected from two 
sources: school records and student self- 
reports. Previously, NAEP has used student 
self-reported race as the primary^ race/ 
ethnicity reporting variable. Starting in 2002, 
school-recorded race has become the race/ 
ethnicity variable presented in N AE’P 
reports. The mutually exclusive racial/ 
ethnic categories are Wliite, Black, Hispanic, 
Asian/Pacific Islander, ^-Vmerican Indian 
(including Alaska Native), and Other. When 
a school reports a student’s race as “Other,” 



that category is used. If the school record 
for race is missing for the student, the 
student’s response to the race/ethnicity 
question is then used. If student data are 
missing (i.e. the student did not respond or 
gave multiple responses), then the student is 
coded to the “Other/missing” category. The 
combination of these two sets of student 
categories is used for the “other” category. 
The race/ethnicity tables in this report omit 
the “Other” category because the percent- 
ages were found to be consistently under 
one percent. Information based on student 
self-reported race/ ethnicity will continue to 
be available on the NAEP Data Tool (littp:/ 
/ nces.ed.gov/ nationsreportcard/ naepdata/). 

Type of Location 

In most N/\EP assessments, results are 
reported for students attending schools in 
three mutually exclusive location types: 
central city, urban tringe/large town, and 
rural/ small town. Results for the NAEP 
2002 Trial Urban District Assessment are 
reported for students attending schools in 
one type of location — central city. 

Following standard definitions established 
by the Federal Office of Management and 
Budget, the US. Bureau of the Census (see 
http:/ /www.census.gov/) defines “central 
city” as the largest city of a Metropolitan 
Statisriciol ^Acea (MSA) or a Consolidated 
Metropolitan Statistical ^Aea (CMSA). An 
MSA is an area defined by the federal 
government for the purposes of presenting 
general-purpose statistics for metropolitan 
areas. Typic:illy, an MSA contains a city with 
a population of at least 50,000 and includes 
its adjacent areas. An MSA becomes a 
CMSA if it meets the requirements to 
qualify as a metropolitan statistical area, has 




111 2002, llie firsi-siage sampling units are public schools in the selectiou oi* tlie combined sample. Further details 
about the procedure for deterniiuing mininuun ssunple size will appear in teclmical documentation section of the 
NAEP web site at http://nces.ed.gov/uatiousreportcarcL 
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a population of 1,000,000 or more, its 
component parts vure recognized as primar\^ 
metropolitan statistical areas, and local 
opinion favors the designation. 

In the NCES Common Core of Data 
(CCD) locale codes are assigned to schools. 
For the definition of central city used in this 
report, two locale codes of the survey are 
combined. The definition of each school’s 
type of location is determined by the size 
of the place where the school is located and 
whether or not it is in an MSA or CMSA. 
School locale codes are assigned by the 
U.S. Bureau of the Census (see http:/ / 
www.census.gov/). For the definition of 
centriil city Kh\EP reporting uses data from 
two CCD locale codes: large city (a central 
city of an MSA or CMSA with the city 
having a population greater than or equal to 
250,000) and midsize city (a central city of a 
MSA or CMSA having a population less 
than 250,000). Central city is a geographical 
term and is not synon}Tnous witli ‘‘inner city.” 

The boundaries of an urban school 
district ;ind a city may not always coincide. 
Los Angeles Unified, for example, extends 
beyond the city boundaries and includes 
urban fringe areas of the MSA, although the 
entire district is coded as central city. Most 
of the other districts included in this report 
have school districts that share the same 
boundaries as the city. The interested reader 
may view the School District Demographics 
website at http:/ /nces. ed.gov/suiweys/sdds, 
where the school district’s boundaries can 
be shown on a map that also has county 
boundaries. In the Houston and Los ^\nge> 
les districts, some students attended schools 
located in the category of urban fringe/ 
large town. These included 6 percent and 
19 percent of fourth-grade students in 



Houston and Los Angeles respectively, as 
well as 24 percent of grade 8 students in 
Los Angeles. Urban fringe /large town is a 
N/VEP classification that combines three 
categories: urban fringe of large city, urb;in 
fringe of midsize city, and large town. iVn 
urban fringe includes all densely settled 
places within MSAs that are classified as 
urban by the U.S. Census Bureau. A large 
town is defined as a place outside MSAs 
with a population of less than 25,000 but 
greater than or equal to 2,500. Across the 
total sample for the N^AEP reading assess- 
ment in 2002, 28 percent of students 
attended schools classified as central city, 42 
percent attended schools classified as urban 
fringe/large town, and 30 percent attended 
schools classified as rural. 

Eligibility for 

Free/Reduced-Price School Lunch 

Based on available school records, students 
were classified as either currentJy eligible for 
the free/reduced-price school lunch compo- 
nent of the Department of Agriculture’s 
National School Lunch Program or not 
eligible. Eligibility for the program is deter- 
mined by students’ family income in relation 
to the federally established poverty level. 
Free lunch qualification is set at 130 percent 
of the poverty level, and reduced-price 
lunch qu'alification is set at 170 percent of 
the poverty level. The classification applies 
only to the school ye;ir when the assessment 
was administered (i.e., the 2001-D2 school 
year) *and is not based on eligibility in 
previous years. If school records were not 
available, the student was classified as 
“Information not available.” If the school 
did not participate in the program, all 
students in that school were classified as 
“Information not available.” 
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